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Overview 


The prospect of a well-paying job for a worker without a college education has significantly dimmed 
in the past three decades, in the wake of sweeping changes in the U.S. economy and labor market. 
The effects of these changes are particularly devastating for young people from disadvantaged urban 
communities. In response to this issue, the SEED Foundation, founded in 1997 by former manage- 
ment consultants Rajiv Vinnakota and Eric Adler, opened the first public, urban, college-preparatory 
boarding school in the country. The primary mission of the SEED School of Washington, DC 
(SEED DC), is to provide an intensive education program that prepares students from low-income 
and underserved communities for college enrollment and success. 

The SEED school, located in a residential section of southeast Washington, serves approximately 
320 sixth- through twelfth-graders. Students attend school on campus five days a week, arriving on 
Sunday evening and going home on Friday afternoon. The SEED model posits that an alternative 
urban academic environment that puts a high priority on academic excellence and personal devel- 
opment will allow students to succeed through high school and in college. An important facet of the 
SEED model is to surround students with a cadre of adults to support them in preparing for college 
success — including their teachers, school administrators, and the residence hall staff. 

Using the random assignment inherent in the school admissions lottery, this report presents results 
from a six-year evaluation of SEED DC, including both an implementation study — to understand 
how the school operates in practice — and an examination of the impacts of winning admission to 
SEED DC on a broad range of student outcomes. 


Key Findings 

• SEED DC creates a highly supportive environment for its students, with a network of caring 
adults. The school provides students with a wide array of services, ranging from academic sup- 
port to emotional support and relationship-building activities. 

• The school produced significant, positive impacts on students’ standardized test scores and pro- 
ficiency levels — particularly in math — in comparison with outcomes among students who did 
not win admission to SEED. 

• For students in the earliest cohorts, who can be followed through high school, SEED DC did not 
increase the proportion who graduated from high school in four years. 

• Although SEED DC showed a couple of positive behavioral effects, it did not show an impact 
on the key nonacademic outcomes, such as teen pregnancy or interaction with the criminal jus- 
tice system, that could justify its higher cost. 

It is important to note that SEED DC operates in a district with many innovative alternatives and, 
owing to local rules, is unable to target the most underserved students. Thus the evaluation cannot 
speak to the question of whether SEED’S unusual boarding school model could produce larger ef- 
fects in different environments, serving students who face more serious obstacles to success. 
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Preface 


Of the 24 million adolescents in the United States today, 41 percent live in low-income families 
and 19 percent, or 4.7 million, live in families with incomes falling below the federal poverty 
threshold. These young people are more likely to move multiple times, to face food insecurity, 
and to live in neighborhoods characterized by crime and a lack of resources. While educational 
attainment is often heralded as a pathway out of poverty, many of these young people live in 
neighborhoods with failing schools and come from families without a history of high school 
completion or college enrollment. 

Education refonners working in low-income areas have tried for decades to improve 
students’ school options and break the cycle of poverty. While there have been some successes, 
for some students, the compounding effects of multiple disadvantages are often too disruptive. 
In response the founders of SEED boarding schools sought to create a holistic intervention that 
provides students with a constant, safe place to live; regular healthy meals; and the kinds of re- 
sources — such as a library and a peaceful outdoor area — found in middle- and high-income 
communities. Within this context SEED strives to provide its students with a rigorous, college- 
ready academic program and supplements it with a youth development-focused life skills cur- 
riculum after school hours. 

The SEED Foundation created its first school, SEED DC, in 1997. It currently serves 
320 students in grades 6 through 12; students live on campus Monday through Friday and go 
home on the weekends. The evaluation described in this report, funded under the Social Innova- 
tion Fund, takes advantage of lotteries within the SEED DC admissions process to assess 
SEED’S effects on students throughout middle and high school. The study focuses mainly on 
students’ academic outcomes but also uses a survey to measure whether SEED students are 
more likely to be engaged in school and have positive plans for the future. 

Notably, SEED improved students’ scores on standardized tests, but to date a partial 
sample of students who won a lottery to attend SEED were no more likely to graduate in four 
years than students who lost the lottery and attended other schools. There is no evidence that 
SEED students were less likely to engage in risky behaviors. Finally, many students left SEED 
after middle school. Still, it will be important to understand whether the gains in academic per- 
fonnance translate into additional high school graduates and higher rates of college enrollment 
in later years. The results raise the question of whether the intensive SEED approach would be 
more effective if aimed more narrowly at students facing very serious obstacles to success. 

Gordon L. Berlin 

President, MDRC 
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Executive Summary 


Sweeping changes in the U.S. economy and labor market over the past three decades have dra- 
matically reduced the availability of well-paying jobs for workers without postsecondary educa- 
tion. Yet one in five high school freshmen nationwide do not graduate in four years, and many 
who do complete school are not ready to perform college-level work. 1 These patterns are particu- 
larly pronounced in urban areas, and among students from low-income and underserved families. 

In recent years, charter schools, which receive public funding but operate independently 
of local school districts, have increased in number and popularity, in part due to their flexible 
governance structure, which allows them to implement innovative new education models. One 
such model belongs to the SEED School of Washington, DC (SEED DC), the nation’s first ur- 
ban, public, college-prep boarding school. The school provides students with an intensive, fully 
integrated academic and boarding school program, including scheduled study time, constant 
access to positive role models, and life skills training. SEED’S model is based on the assump- 
tion that, for certain disadvantaged students who face overwhelming barriers to success at home 
and in the community, piecemeal reform efforts will not be sufficient. 

This report presents the findings from a rigorous evaluation of SEED DC, which was 
supported by the Social Innovation Fund (SIF), a program of the Corporation for National and 
Community Service (CNCS). The Edna McConnell Clark Foundation (EMCF) is leading a SIF 
project that includes support from CNCS and 15 private co-investors. EMCF’s SIF project in- 
cluded an investment in the SEED Foundation, the national nonprofit organization that oversees 
SEED DC and the two other SEED schools (in Baltimore and Miami) currently in operation. 

SEED has been the subject of previous studies, including an impact study that found 
that SEED led to significant gains in standardized test scores in seventh and eighth grade. 2 The 
authors of that study questioned whether these increases were large enough to justify the high 
cost of the boarding school model. If SEED affects nonacademic outcomes such as teen preg- 
nancy or crime involvement, which trigger very high social costs, the program could turn out to 
be a worthwhile investment of public funds. 

The study uses SEED DC’s annual admissions lottery to identify two comparable 
groups of students: those who applied to SEED and were selected, at random, to be offered a 


'Grace Kena, Susan Aud, Frank Johnson, Xiaolei Wang, Jijun Zhang, Amy Rathbun, Sidney Wilkinson- 
Flicker, and Paul Kristapovich, The Condition of Education 2014, NCES 2014-083 (Washington, DC: National 
Center for Education Statistics, 2014). 

2 Vilsa E. Curto and Roland G. Fryer Jr., “Estimating the Returns to Urban Boarding Schools: Evidence 
from SEED,” National Bureau of Economic Research Working Paper No. 16746 (2011). 
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slot in the school, and those who applied to SEED and were not offered a slot. By following 
those two groups of students over time, the study can estimate the impacts of SEED DC on 
standardized test scores, high school graduation rates, and other nonacademic outcomes. This 
study focuses only on SEED DC, which is by far the most mature of the existing SEED schools. 

The evaluation examined both the implementation of SEED DC and its impact on stu- 
dent outcomes. The study’s two overarching research questions are as follows: 

• How is SEED DC structured and how does it operate in practice? Using 
interviews with staff members and students, observation, and other methods, 
the evaluation team set out to understand the on-the-ground reality of SEED 
DC and how the school is experienced by students. This information not only 
helps in interpreting the impact findings, but also may help SEED improve 
service delivery over the long run. 

• What is the effect of being offered an opportunity to attend SEED DC on 
student outcomes? Academic outcomes include standardized test scores and 
high school graduation. Nonacademic outcomes include both attitudes (for 
example, college aspirations) and behaviors (positive ones like homework 
completion and risky ones like alcohol and drug use). 

The evaluation focuses on 766 students who “won” or “lost” the SEED lottery as fifth- or sixth- 
graders between 2006 and 2011. The study followed those students through the 2013-2014 aca- 
demic year, which means that only a small number of them could have graduated from high 
school or enrolled in college during the study period. Thus, while improving students’ perfor- 
mance in college is a key goal of SEED’S, it is too soon to assess whether SEED improves stu- 
dents’ postsecondary outcomes. 


SEED DC in Operation 

Located in a residential section of southeast Washington, SEED DC serves approximately 340 
sixth- through twelfth-grade students. Students in the study sample were primarily African- 
American and were economically and academically disadvantaged. Of those who won the 
SEED lottery (the SEED group), four out of five qualified for free or reduced-price lunch. In the 
year they applied to SEED, 14 percent of the SEED group students qualified for special educa- 
tion services and just under 50 percent scored at or above proficient on the district-wide reading 
and math exams. 

Students attend school on campus five days a week, arriving on Sunday evening and re- 
turning home on Friday afternoon. During the days that students reside on campus, they have 
access to quiet places to study and sleep, nutritious meals, academic resources, and spaces to 
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engage in extracurricular activities. By design, students are also surrounded by a cadre of caring 
adults who support them in preparing for success in college. In both academic and residential 
life programs, the school uses a grade-based cohort system, meaning that distinct goals, expec- 
tations, and approaches are set for middle school, ninth-grade, and high school students. 

Academic Curriculum 

The school philosophy is that all students have the same inherent potential for academic 
success, and thus all students are expected to excel at SEED. The academic department strongly 
believes in using data to guide and inform instruction, so all students take interim assessments in 
English and math four times per year. After each assessment, academic and Student Life staff 
members meet in teams to discuss the results and identify priority issues. Teachers then use the 
assessment results to develop lesson plans to “reteach” skills that students have been unable to 
master. 


SEED students are expected to attend college following high school graduation. The 
College Counseling department is responsible for providing support to students in the college 
search, application, and selection process. From sixth grade on, students are encouraged to visit 
the College Cafe, a colorful and inviting space stocked with infomiation and decorated with 
memorabilia from many of the nation’s colleges. In middle school, students engage in discus- 
sions about the value of enrolling in college, visit a college campus, and participate in activities 
to strengthen their academic habits. Starting in ninth grade, students practice taking college en- 
trance exams (the PSAT) so that they become familiar with the test and can improve their 
scores. High school students have access to test preparation materials during after-school hours. 
Eleventh-graders receive college advising focused on finding the “right-fit” college — the one 
that is the best academic match and also meets their financial, social, and personal needs — and 
students in the twelfth grade are actively engaged in the college search, application, and choice 
process. 


A unique feature of the SEED student experience is the support that SEED students and 
graduates receive from the College Transition and Success (CTS) team, a unit within the SEED 
Foundation. Working in collaboration with the College Counseling Department, the CTS team 
holds a series of college transition workshops for seniors and their parents (including a financial 
literacy workshop) and helps students finish required college enrollment paperwork. The CTS 
Team also monitors and supports SEED graduates as they make the transition to college and 
maintains contact with SEED graduates who are enrolled in college. 

Student Life Curriculum 

Perhaps the most distinctive aspect of SEED’S learning environment is the time that 
adults spend with students after school and through the evening. The Student Life Department is 
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responsible for developing and coordinating residential life programming and managing stu- 
dents’ time outside of the traditional academic day. Students are organized into houses (or 
groups) within the dorms, each of which is led by a resident adviser. Each house is named after 
a college or university and decorated with its pennants, pillows, and the like. 

The Student Life Department aims to develop students’ behavioral, social, and life 
skills while reinforcing what they are learning in the classroom. Middle school programming is 
intended to develop and refine social skills that are connected to meeting behavior expectations 
and routines, such as following instructions the first time they are given, adhering to the school 
dress code, and learning how to disagree appropriately. Ninth-grade Student Life activities aim 
to develop and reinforce the skills and habits necessary for success in high school, such as plan- 
ning ahead, using anger control strategies, and building strong self-esteem. High school pro- 
gramming focuses on the transition to college. To accomplish these goals, the majority of Stu- 
dent Life time is structured, especially for the middle school grades. 

The SEED-created Habits for Achieving Life-Long Success (HALLS) program teaches 
students social and basic life skills, such as decision making and communication strategies, and 
the importance of taking responsibility for oneself and others. HALLS activities focus on a va- 
riety of topics, such as bullying, dating relationships, and appropriate dining etiquette. 

Students described the social and life skills they are being taught as an important step in 
their preparation for college and beyond. They told of instructional and noninstructional staff 
members routinely discussing the personal habits and skills that students need to succeed in col- 
lege, such as self-motivation, discipline, independence, strong time management skills (a recur- 
ring theme in all interviews and focus groups), leadership qualities, and other personal charac- 
teristics. Middle school students characterized their schedules as being regimented and related 
an overall sense of being overscheduled, yet many also reported taking advantage of the various 
extracurricular activities offered at SEED. For example, some participated on the track team and 
some in student government. 


Impacts on Academic Achievement and Behavior 

At the beginning of the study period, students entered SEED as seventh-graders, but two years 
later SEED changed its model and began to enroll sixth-graders as well. In 2010-201 1, SEED 
fully transitioned to its new model and admitted students only as sixth-graders. Analyses drew 
on separate samples for sixth-grade and seventh-grade entrants to estimate the short-term effects 
of SEED and drew on the sample of students from the first two years of the study (cohorts 1 and 
2 of the seventh-graders) to estimate the longer-term effects. 
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Washington, DC, is an especially charter-rich environment, and many of the students 
who applied to SEED but did not win the lottery actively sought out other innovative school 
options. Specifically, of the SEED lottery losers, roughly half the students enrolled in charter 
schools and half enrolled in traditional District of Columbia middle schools. Thus, in this study 
the SEED student experience was compared with a diverse set of other school experiences that 
may not exist in other contexts. 

Short-Term Effects of SEED 

About 20 percent of the students who won the SEED lottery (the SEED group) did not 
enroll in SEED the following fall. Among lottery winners who did not enroll in SEED, a little 
more than half enrolled in other charter schools and the remainder enrolled in traditional District 
of Columbia middle schools. 

Short-term academic effects were measured by standardized test scores in the first two 
years of follow-up. Findings for the Seventh-Grade Entrant Sample are as follows: 

• On average, being offered the opportunity to attend SEED increased students’ 
academic achievement in math. In the first year, SEED group scores were 
higher than non-SEED group scores by 0.24 standard deviation, which is 
roughly equivalent to a 76 percent improvement on top of the typical annual 
gains for this age group. In the second year, the SEED effect in math was 
equivalent to one and a half years of typical growth. 3 

• Students in the SEED group did not perform better than students in the non- 
SEED group in their first year of follow-up on the standard reading exam, but 
in the second year, the SEED group’s test scores exceeded those of the non- 
SEED group by the equivalent of one year of typical growth in reading. 4 

Students in the Sixth-Grade Entrant Sample experienced a similar, though slightly 
weaker, pattern of positive SEED effects. 

Longer-Term Effects of SEED 

In their third follow-up year, students in the first two cohorts made the transition into 
high school. 5 SEED’S high school model is different from the middle school model in a few key 
ways — boys and girls are taught in the same classroom, students’ time after school is less 
structured, and students begin more rigorous college-preparatory activities. Between the second 

3 Carolyn J. Hill, Howard S. Bloom, Alison Rebeck Black, and Mark W. Lipsey, “Empirical Benchmarks 
for Interpreting Effect Sizes in Research,’’ Child Development Perspectives 2, 3 (2008): 172-177. 

4 Hill, Bloom, Black, and Lipsey (2008). 

5 Ten percent of the students were retained in grade. 
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and third follow-up years, roughly 20 percent of the SEED lottery winners in cohorts 1 and 2 
who were enrolled in SEED chose to leave and attend a different high school. 

In the fall of 2013 the research team surveyed lottery winners and lottery losers in co- 
horts 1 and 2 and asked them a variety of questions about their experiences in school. At that 
point, most of the respondents were high school seniors or had recently graduated. Overall, stu- 
dents in the SEED group reported experiencing a more rigorous and supportive academic envi- 
ronment. 

• Students in the SEED group took fewer Advanced Placement courses but a 
larger number of foreign language, advanced science, and advanced math 
courses; participated in more extracurricular activities; received more aca- 
demic support from teachers and adults; and participated in more college-prep 
and work-readiness activities. 

• SEED group students reported having more orderly classrooms and more aca- 
demically motivated peers. Students did not report a difference in the calm- 
ness and order of weekday living enviromnents or the frequency of meals, 
though students in the SEED group did report that they slept less and exer- 
cised more during the week. 

The effects on high school graduation for the earliest cohorts were limited: 

• Being offered the opportunity to attend SEED did not have an effect on stu- 
dents’ probability of four-year high school graduation. 

In addition to positively affecting students’ academic achievement, SEED aims to pro- 
mote positive behaviors like rigorous study habits and self-control, while discouraging “risky” 
ones like alcohol use and unplanned pregnancy. SEED may have had small effects on some 
measures of student behavior, but only four are statistically significant: 

• While the SEED group did not report having developed more rigorous study 
habits or organizational skills than the non- SEED group, they did report 
spending four more hours a week doing homework than the non- SEED group. 

• Students in the SEED group reported slightly lower tobacco use in the past 30 
days than non- SEED group students. 

• Students in the SEED group indicated slightly more frequent risky behavior 
(for example, skipping school, arguing with parents, or hitting someone) in 
the three months before they were surveyed, compared with non-SEED group 
students. 
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SEED group students reported slightly lower levels than non- SEED group 
students on scales designed to measure “grit” or perseverance. 


Conclusions 

Several factors are critical to the interpretation of the findings to date from this evaluation: 

• Cost. Owing to its boarding school model, SEED costs at least twice as much 
per student as a traditional nonresidential school. If SEED has impacts on 
four-year high school graduation and nonacademic outcomes such as teenage 
childbearing and justice involvement, it would have the potential to produce 
large societal benefits that would offset its cost. At this point, there is little ev- 
idence that SEED DC has affected either graduation or the nonacademic out- 
comes, though it is important to note that the sample size for measuring long- 
er-term impacts (about 200 students) is quite small. 

• Enrollment targeting. SEED’S founders argue that the intensive, holistic, 
boarding-school model is needed for some students who face very serious ob- 
stacles to school success in their homes and communities. And, indeed, both 
the Maryland and Florida SEED schools are open only to students who meet 
certain specific criteria signifying severe disadvantage. In contrast, owing to 
local rules, the DC school, the focus of this evaluation, is open to any student 
who resides in Washington. It is possible that SEED’S model would produce 
larger impacts for students facing more serious obstacles to success. 

• Local educational context. Washington, DC, has many innovative charter 
and magnet schools, and it appears that a large proportion of the students who 
lost the SEED lottery enrolled in these schools, particularly by the time they 
reached high school, when two-thirds of the non-SEED group were attending 
charter or magnet schools. It is possible that SEED would make a bigger dif- 
ference in a context with fewer innovative alternatives. 

• Length of stay at SEED. SEED DC’s impacts on academic proficiency are 
substantial, particularly in the middle school years, and other literature sug- 
gests that middle school test scores are highly predictive of high school grad- 
uation. Yet there is no evidence to date that SEED has increased high school 
graduation rates. This fact, coupled with data showing that less than half the 
SEED group was still at the school in twelfth grade, raises the question of 
whether SEED could have larger, more sustained impacts if more students 
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remained in the school longer and received a larger “dose” of SEED — in 
other words, whether the school needs to focus more on promoting retention. 

• Implementation quality. While the implementation study noted many posi- 
tive qualities of SEED DC’s operation, it also raised questions about the 
quality of instruction in SEED DC’s classrooms — particularly at the high 
school level — and noted that many students seemed to be struggling with the 
transition from eighth grade to ninth grade, even while remaining at the same 
school. The SEED Foundation has recently announced a renewed and intensi- 
fied focus on the quality of instruction, leadership, and services at its schools. 
For example, the foundation reports that SEED DC has identified new curric- 
ulum resources to strengthen the middle school math program and is seeking 
to increase student engagement through interactive learning technologies. 
Given these and other ongoing changes, it is possible that impacts on student 
outcomes — and, perhaps, retention rates — will be stronger in the future. 
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Chapter 1 

Introduction 


Sweeping changes in the U.S. economy and labor market over the past three decades have dra- 
matically reduced the availability of well-paying jobs for workers without postsecondary educa- 
tion. Yet one in five high school freshmen nationwide do not graduate in four years, and many 
who do complete school are not ready to perform college-level work. 1 These patterns are particu- 
larly pronounced in urban areas, and among students from low-income and underserved families. 

With these trends in mind, policymakers, practitioners, and researchers have developed 
and promoted a variety of approaches to improving students’ high school success. Some inter- 
ventions are implemented within existing schools, such as Talent Development, which bolsters 
the support students receive while transitioning from eighth to ninth grade. 2 Others restructure 
schools to provide personalized learning environments that foster relationships between teachers 
and students and closely track students’ academic progress (Ninth-Grade Academies) and oth- 
ers create new district schools altogether (New York City’s Small Schools of Choice). 3 In recent 
years, charter schools, which receive public funding but operate independently of local school 
districts, have increased in number and popularity, in part due to their flexible governance struc- 
ture, which allows them to implement innovative new education models. 

Rigorous studies have found positive results for some of these new approaches, but 
much remains to be done. For example, in a 2010 report on the impacts of 36 charter middle 
schools across 15 states, on average, students who applied to and won the lotteries for oversub- 
scribed charter schools did not perform higher on follow-up reading and mathematics exams 
than students who did not win the lotteries. 4 More encouraging were findings from the research- 
ers’ secondary analysis: Among schools serving more low-income or low-achieving students, 
charter schools did have positive effects on students’ math scores, although not on students’ read- 
ing scores. Findings from MDRC’s study of New York City’s Small Schools of Choice also in- 
dicate a need for more intensive interventions for the most disadvantaged students. Specifically, 
while enrolling in one of these schools substantially increased graduation rates for the lowest- 
achieving students, less than half the study students in this group graduated within four years. 5 


'Kena et al. (2014). 

2 Kemple (2008); Kemple, Herlihy, and Smith (2005); Snipes, Holton, Doolittle, and Sztejnberg (2006). 
3 Bloom and Unterman (2014). 

4 Gleason et al. (2010). 

5 Bloom and Unterman (2014). 


1 



This report presents the findings from a rigorous evaluation of the SEED School of Wash- 
ington, DC (SEED DC), the nation’s first urban, public, college-preparatory boarding school. 
SEED’S boarding school model is predicated on the assumption that, for certain disadvantaged 
students who face overwhelming barriers to success at home and in the community, school re- 
forms such as those described above, and other enhancements such as after-school programs, 
will not be sufficient. SEED’S leaders believe that, for these students, achieving success in high 
school and beyond requires a fully integrated academic and boarding program that also provides 
scheduled study time, constant access to positive role models, and life skills training. 

This report is based upon work supported by the Social Innovation Fund (SIF), a pro- 
gram of the Corporation for National and Community Service (CNCS). The Edna McConnell 
Clark Foundation (EMCF) is leading a SIF project that includes support from CNCS and 15 
private co-investors. EMCF’s SIF project included an investment in the SEED Foundation, the 
national nonprofit organization that oversees SEED DC and the two other SEED schools (in 
Baltimore, Maryland, and Miami, Florida) currently in operation. The evaluation is being con- 
ducted by MDRC, a nonprofit, nonpartisan education and social policy research organization. It 
uses SEED DC’s annual admissions lottery to identify two comparable groups of students: 
those who applied to SEED and were selected, at random, to be offered a slot in the school, and 
those who applied to SEED and were not offered a slot. By following those two groups of stu- 
dents over time, the study can estimate the impacts of SEED DC on standardized test scores and 
high school graduation rates, as well as nonacademic outcomes. This study focuses only on 
SEED DC, which is by far the most mature of the existing SEED schools. 

In sum, the study found that SEED DC creates a highly supportive environment for its 
students. The program has produced notable, positive increases in students’ standardized test 
scores and proficiency levels. But, for the portion of students it was able to follow through all 
four years of high school, the study did not find an impact on the proportion of students who 
graduated from high school in four years or on key nonacademic outcomes. 

About SEED 

The SEED Foundation was created in 1997 by Rajiv Vinnakota and Eric Adler, two former 
management consultants who left their jobs to create a public boarding school for low-income, 
disadvantaged students. The SEED School of Washington, DC, opened in 1998 as a public 
charter school and currently serves about 320 students in grades 6 through 12. 

As the first SEED school, SEED DC has received widespread national attention. In 
2005, SEED received the Innovations in American Government award from Harvard’s John F. 
Kennedy School of Government. In 2009, President Obama visited the DC school to sign the 
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Edward M. Kennedy Serve America Act. In 2010, SEED was featured on 60 Minutes and in the 
documentary film Waiting for “Superman. ” 

Ten years after SEED DC opened, a second SEED school was created in Baltimore, 
serving students throughout the state of Maryland. SEED Maryland serves about 400 students in 
grades 6 through 12. The third school, in Miami, admitted its first group of 60 sixth-graders in 
2014. SEED DC is open to anyone who lives in the District of Columbia, while the Maryland 
and Florida schools target students who meet specific criteria signifying serious disadvantage. 
In late 2014 the SEED Foundation announced a comprehensive effort to improve the quality of 
academic and student support programming at all its schools. The SEED Foundation reports 
that the new approach is being implemented first in the new Miami school. For more infor- 
mation about that school and its early success, see Appendix A. 


The SEED Evaluation 

SEED has been the subject of several qualitative studies and one impact study, conducted by 
Vilsa Curto and Roland Fryer Jr. of Harvard University. 6 Like the current evaluation, Curto and 
Fryer’s study built on the admissions lottery for SEED DC. It found that SEED led to signifi- 
cant gains in standardized test scores in seventh and eighth grade, but the authors questioned 
whether these increases were large enough to justify SEED’S very high cost — more than 
$35,000 per year per student. They noted that other charter schools have produced similar re- 
sults without the boarding component, which accounts for a large proportion of SEED’S cost. 

The MDRC evaluation builds on the Curto and Fryer analysis by following a larger 
number of students for a longer period, conducting an implementation study to more fully de- 
scribe how SEED operates, and obtaining data on a broader range of outcomes from school rec- 
ords and a student survey. This latter point is critical, because if SEED affects nonacademic 
outcomes such as teen pregnancy or crime involvement, which trigger very high social costs, 
the program could turn out to be a worthwhile investment of public funds. 

The design for MDRC’s evaluation of SEED DC was shaped by the logic model shown 
in Figure 1.1. Starting at the far left, the model’s inputs include the students, teachers and staff, 
the boarding school infrastructure, and support from the SEED Foundation, among others. The 
inputs lead to a set of specific activities and program components: the academic program, the 
Student Life program, College Counseling, and Student Support Services. The model hypothe- 
sizes that these inputs and program components, if implemented according to plan, will lead to a 


6 Curto and Fryer (2011). 
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Inputs 

Students and families 

School leaders and 
trustees 

Teachers and staff 

SEED program model 
and philosophy 

Support from the 
SEED Foundation 

Community partners 

Donors and investors 
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Figure 1.1 

SEED DC Logic Model 

Program Components Intermediate Outcomes 


Academic program 


Student Life program 


College Counseling 


Student Support 
Services 


Academic Outcomes 
Improved standardized test scores 
Improved GPA 

Increased middle school reading levels 
High AP participation and scores 
College admission 

Social Outcomes 
Increased “college knowledge” 
Reduced teen pregnancy 
Lower involvement in juvenile justice 
system 

Increased knowledge of life skills 

Well-Being Outcomes 
Positive school climate 
High amount of family involvement 
Improved mental health 
Low frequency of disciplinary actions 
High student and staff satisfaction 


Community characteristics 


Long-Term Outcomes 


High school 

graduation College 

and college graduation 

matriculation 


SOURCE: Adapted from SEED Foundation. 

NOTE: GPA = grade point average; AP = Advanced Placement. 








different type of school environment that is orderly and highly supportive, and to the transmis- 
sion of specific knowledge and values to students. This result, in turn, should produce both in- 
termediate student achievement outcomes — such as better grades and test scores — and well- 
being outcomes, as well as longer-term student outcomes such as high school graduation, and 
ultimately, college success. 

In line with this logic model, the evaluation studied both the implementation of SEED 
DC and its impact on student outcomes. Owing to its timing, however, the evaluation cannot 
assess whether SEED is achieving its longer-term goals, such as helping students succeed in 
college. The evaluation’s two overarching research questions are: 

• How is SEED DC structured and how does it operate in practice? Using 
interviews with staff members and students, observation, and other methods, 
the evaluation team set out to understand the on-the-ground reality of SEED 
DC and how the school is experienced by students. This information not only 
will help in interpreting the impact findings, but also may help SEED im- 
prove service delivery over the long run. 

• What is the effect of being offered an opportunity to attend SEED DC 
on student outcomes? The primary outcomes in the study are academic: 
standardized test scores and high school graduation. Secondary, nonacadem- 
ic outcomes include both student attitudes (for example, college aspirations) 
and student behaviors (for example, positive ones such as homework com- 
pletion and risky ones such as interactions with the criminal justice system or 
unplanned pregnancy). Because some lottery winners did not enroll in SEED 
DC, a secondary analysis explores how enrolling in the school affects stu- 
dents’ short- and longer-term outcomes. 

The evaluation focuses on 766 students who “won” or “lost” the SEED lottery as fifth- 
or sixth-graders between the 2005-2006 and 2010-2011 school years. The study followed those 
students through the 2013-2014 academic year, which means that only a small number of them 
could have graduated from high school or enrolled in college during the study period. Thus, 
while improving students’ performance in college is a key goal of SEED’S, it is too soon to as- 
sess whether SEED improves students’ postsecondary outcomes. 

The remaining chapters of this report describe the findings from the SEED evaluation. 
Chapter 2 describes how SEED DC is staffed and organized and how it operated in practice 
over the course of three school years (starting in fall 201 1). This chapter draws primarily from a 
series of site visits to SEED, which included interviews with staff members and students and 
observations of classes and other school activities. The research team also conducted interviews 
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with officials from the SEED Foundation. In essence, these chapters focus on the left side of the 
logic model, describing both the SEED model and its implementation. 

Chapter 3 addresses the second research question. It focuses both on the “treatment con- 
trast” — how the experiences of lottery winners and lottery losers differed — and on the out- 
comes that students achieved. The analysis uses data from school records and a student survey 
to assess both academic and nonacademic outcomes. Chapter 4 summarizes the main conclu- 
sions and discusses their implications. 
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Chapter 2 

SEED DC in Operation 


As described in Chapter 1 , the SEED model posits that a different type of school environment 
can produce positive outcomes for students. In order to provide context for the impacts that will 
be discussed in the following chapters, it is necessary to understand what this different learning 
enviro nm ent looks like on the ground. This chapter describes the SEED model in practice based 
on interview, focus group, and observational data collected in May 2012, November 2012, and 
November 2013. The study does not focus on fidelity of implementation — that is, whether 
SEED DC programming was implemented according to the SEED model’s design — but rather 
describes the school as it appeared in operation. The analytic approach and data collection activ- 
ities are described in greater detail in Appendix B. 


General Structure of SEED DC 

Located in a residential section of southeast Washington, DC, in Ward 7, SEED DC is a col- 
lege-preparatory public charter boarding school that serves approximately 320 sixth- through 
twelfth-grade students. Students attend school on campus five days a week, arriving on Sunday 
evening and returning home on Friday afternoon. The school’s physical facilities include an 
academic center, where classes are held; an administrative building; a library; a gymnasium; a 
college counseling office (the College Cafe); and two single-gender dormitories. Each floor of 
the dormitories is equipped with a common room, bathroom facilities, a small reading room, 
and small bedrooms that are furnished with beds, closets, desks, and chairs to accommodate two 
students each. During the five days that students reside on campus, they have 24-hour access to 
quiet places to study and sleep, nutritious meals, academic resources, and spaces to engage in 
extracurricular activities. By design, students are also surrounded by a cadre of caring adults 
who will support them in preparing for success in college. 

SEED DC is led by a head of school responsible for keeping the school’s operation true 
to its mission. While this chapter concentrates on the rest of the staff responsible for student 
programming, it is important to note that the school is supported by a number of other adminis- 
trative staff, including a managing director and directors of finance, human resources, develop- 
ment, and campus operations. 

A distinguishing feature of SEED DC programming is that the school’s goals, expecta- 
tions, and curricula are designed specifically for three groups of students: middle school stu- 
dents, ninth-graders, and tenth- through twelfth-graders. While ninth grade is technically part of 
SEED DC’s high school, the model treats this year as a uniquely important transitional year 
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warranting its own specific focus. A second distinguishing feature of the SEED model is how 
school programming is organized in terms of content area: Besides academics, there are de- 
partments of Student Life, Student Support Services, and College Counseling (see Figure 1.1). 
While none of the four departments functions independently of the others, each has its own spe- 
cific role and responsibilities. More detail on the staffing structures of each department is in- 
cluded in the following sections, as well as in the organizational chart shown in Figure 2.1. The 
rest of the chapter presents information organized into these four areas, as well as a brief section 
explaining how they interact. 


Academics 

The school principal is the leader of the academic department, supported by teachers and middle 
managers focused on middle school- and high school-specific curricula, special education, and 
evening programming. For more detail on staffing in SEED DC’s academic department, see 
Figure 2.1. SEED DC’s school day mirrors the schedule of a typical school day: seven to eight 
hours of instruction divided into subject-specific class periods. Tables 2.1 and 2.2 list the middle 
and high school courses offered by grade. 

The school philosophy is that all students have the same inherent potential for academic 
success, and thus all students are expected to excel at SEED. The academic department strongly 
believes in using data to guide and inform instruction, so all students take interim assessments in 
English and math four times per year. After each assessment, academic and Student Life staff 
members meet in teams to discuss the results and identify priority issues. Teachers then use the 
assessment results to develop lesson plans to “reteach” skills that students have been unable to 
master. 


The SEED DC academic program follows the school’s grade-based cohort system, 
meaning that distinct goals, expectations, and approaches are set for middle school, ninth-grade, 
and high school students. More detail on SEED’S approach to middle school and high school 
(including ninth grade) academics is included in the following sections. 

Middle School 

Middle school administrators and faculty and staff members reported that many sixth- 
grade students enter SEED DC reading at one or more levels below grade level. We present base- 
line test score data later in the report. They also suggested that over time, SEED has observed an 
increase in the number of incoming students eligible for special education. To respond to stu- 
dents’ needs, the middle school program is intentionally designed to remediate gaps and pro- 
mote growth. Diagnostic testing is administered on a frequent basis so that data can inform in- 
dividualized instruction. An administrator explained that the school “make[s] sure that kids are 
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SEED DC Organizational Chart 
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Special Education 
Coordinator 


HS Program 
Director 


MS Program 
Director 
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Director 
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Coordinator: HS 
programming 
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Dean 


Assistant 

Deans 


SOURCE: SEED Foundation. 


NOTES: HS = high school; MS = middle school; LSC = life skills counselor; RA = resident adviser. Organizational chart omits administrative staff not involved with student 
programming. 
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Table 2.1 

Middle School Courses at SEED DC 


Subject 

6th grade 

7 th grade 


8th grade 

English and language arts 

Reading and Writing Reading and Writing 

Reading and Writing 

Math 

Foundations of Math Prealgebra 


Algebra 

Science 

Earth Science 

Life Science 

Physical Science 

Social studies 

U.S. Geography 

Civics 


World Geography 

Foreign language 




Introduction to Spanish 

SOURCE: Interviews with SEED DC administrators. 
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Table 2.2 




High School Courses at SEED DC 


Subject 

9th grade 

1 0th grade 

1 1 th grade 

12 th grade 

English and 
language arts 

Reading and 

Writing 

Reading and 

Writing 

AP English or 
English II 

AP Literature or 

12th-grade English 

Math 

Geometry 

Algebra II 

Pre-Calculus or 

Probability and 
Statistics 3 

AP Calculus or 

Calculus or 
Precalculus b 

Science 

Conceptual 

Physics 

Biology 

Chemistry 

AP Biology or 
Anatomy and 
Physiology 

Social studies 

World History I 

World History II 

AP U.S. History or DC History and 

U.S. History either AP U.S. 

Government or 

U.S. Government 0 

Foreign language 

Spanish I 

Spanish II 

Spanish III 



SOURCE: Interviews with SEED DC administrators. 

NOTES: AP = Advanced Placement. 

“Probability and Statistics is not always offered. 

b Those who advance from Probability and Statistics take Precalculus. 

“Students take one semester of each history course. 
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being given the chance to all get to the same endpoint, but on a path that allows them to grow 
from where they’re starting.” An example of this diagnostic approach is the school’s practice of 
making students aware of their Fountas and Pinnell reading levels — a system that supports 
students in level-appropriate guided reading — so that they can work on advancing during the 
independent reading time that is built into the daily student schedule . 1 

In line with the philosophy that all students can succeed, the school offers a variety of 
interventions to help struggling students master foundational skills. All middle school students 
participate in a daily class period of “targeted instruction,” an opportunity for remediation in 
math or reading, if needed, or for acceleration. 

A distinguishing feature of the middle school is the delivery of instruction in single- 
gender classrooms. Studying the effectiveness of the single-gender approach was not part of the 
evaluation. However, administrators and faculty members indicated in interviews that the 
school chose to implement single-gender classrooms in part because of distractions that can oc- 
cur when middle school boys and girls are placed in the same classrooms. Middle school stu- 
dents interviewed for this study agreed that single-gender classrooms were less distracting than 
coed classrooms. 

Observation findings. The evaluation team conducted a small number of informal ob- 
servations of middle school classrooms to gain a general understanding of how the middle 
school classes are structured, how teachers and students interact, and the types of instructional 
practices being used at the middle school level. The evaluators noted numerous instances of in- 
dependent work being done quietly and effectively. Evaluators also observed positive examples 
of teaching strategies being used. In one instance a teacher encouraged students to use higher- 
order thinking skills by looking at a picture, drawing conclusions based on their observations, 
and defending their reasons for drawing those conclusions. In another instance, evaluators saw 
evidence of coordination among middle school teachers when the U.S. Geography teacher in- 
troduced a new topic, coal, by linking it to a related science lesson on minerals in Earth Science. 

Evaluators also observed a few instances when students were confused by material cov- 
ered in previous classes. For example, in sixth-grade General Foundations of Math, students 
struggled to identify which of two fractions was larger when practicing multiplying and divid- 
ing with improper fractions. In a middle school English class, students were left alone to work 
on a research project and seemed confused about how to differentiate between a main topic and 
subtopics. Unaware of the misunderstanding, the teacher did not address it. 

In focus groups, most middle school students said they were able to keep up with the 
academic course work and that teachers were willing to offer step-by-step instructions to help 


'Fountas and Pinnell (2014). 
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them succeed in the classroom. A few students explained that before enrolling at SEED DC, 
they often felt bored in classes when they were capable of completing advanced course work 
but were not allowed to move ahead. At SEED DC, these students feel more engaged in their 
course work. When discussing their reading levels, two students described the Fountas and Pin- 
ned reading system (of levels A to Z+) as “awesome.” Another said it gave students “something 
to look up to.” 

High School 

Ninth grade. When SEED students make the transition to high school, they take clas- 
ses in coed settings and are expected to demonstrate mastery in all subject areas. In contrast 
with the middle school focus on remediation, students are expected to be at grade level in basic 
reading, writing, and mathematics so that they can build critical reading, analytic thinking, and 
problem-solving skills. 

Because of higher expectations in and outside the classroom and an increase in academ- 
ic rigor in ninth grade, SEED DC views this as a very important transitional year for students. 
To prepare students for the change, the school provides a two- to three-day freshman bridge 
program for students in the summer between eighth and ninth grades. In ninth grade, students 
take all their classes together and are treated as their own grade-specific cohort, separate from 
middle and high school. But students who begin ninth grade below grade level often struggle 
with understanding advanced texts and concepts. Ninth-grade teachers suggested that the transi- 
tion between eighth and ninth grades causes some struggling students to seriously consider 
whether they can succeed at SEED. One ninth-grade teacher offered the following analysis: 

I think our students start to kind of realize, “. . . I can either make it here or I need 
to go. Maybe I’m not going to be able to rise to the occasion in ninth and tenth 
grade.” And I think that’s why we tend to lose more kids around those grades. So 
the ninth grade is very important here . . ., but elsewhere too. The kids start to re- 
alize, “OK, school is for me or school is not for me.” ... I think some of them 
just have trouble making a distinction, “Oh, high school’s not just a continuation 
of middle school, it’s actually a separate entity.” 

Ninth-grade teachers also indicated a need to align the eighth- and ninth-grade curricula 
so that students are better prepared for the transition. Ninth-grade teachers reported working 
with the middle school staff to begin to introduce ninth-grade reading content in the eighth- 
grade curriculum, but they found that the reading gap was too large. Another staff member on 
the academic team reported high failure rates for the ninth-grade cohort in English, writing, ge- 
ometry, and science, and identified the ninth-grade cohort as the cohort most often enrolled in 
on-site summer school. 
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Grades 10 through 12. After ninth grade, the high school curriculum focuses on pre- 
paring students for success in college and beyond by further developing their higher-order 
thinking and problem-solving skills. SEED DC follows the Washington school district’s re- 
quirements for high school graduation and also requires students to earn additional credits. For 
example, SEED DC students must earn three credits of foreign language whereas the district 
requires only two, and all SEED DC science classes are laboratory sciences, whereas the district 
requires only two lab sciences. The school also provides six Advanced Placement classes for 
eligible students: eleventh-grade U.S. History and English Language and twelfth-grade Biology, 
Calculus, Government, and Literature. In addition, there are a few opportunities for eligible stu- 
dents to enroll in college-level courses at a local college. 

Observation findings. The evaluation team visited a small number of high school 
classrooms to obtain a general understanding of the instructional practices being used at the 
high school level and to understand how students are being prepared for college-level work. The 
evaluation team found high school classrooms to be orderly, and most students participated in 
classroom assignments. However, evaluators did not observe many examples of students being 
prepared for challenging college course work that would qualify them for admission to competi- 
tive colleges. For example, the evaluation team did not see significant evidence of students be- 
ing asked to use higher-order thinking skills, such as critical, logical, or reflective thinking, 
when completing classroom assigmnents. The team also observed several missed opportunities 
for redirecting students or correcting misinformation. For example, during individual student 
presentations in a government class, incorrect information shared during a student presentation 
about a legislative bill was not corrected, even when the activity included structured space for 
feedback and corrections. In Spanish II, at no point were students asked to speak or write a full 
sentence in the language. At one point, the teacher chose not to explain a rule, telling the stu- 
dents it was too difficult for them to understand. In addition, while student attention varied from 
class to class, in some instances students appeared to struggle with concentrating on tasks (for 
example, by talking with peers, asking to leave the room, or refusing to participate when called 
upon). 


While the evaluation team found reason to question whether students are being pre- 
pared for higher-tier colleges, high school students reported that they found the curriculum chal- 
lenging. All students interviewed described being overwhelmed by their course loads and con- 
cluded that they were being prepared for college-level work (a few seniors in the focus group 
were enrolled in Advanced Placement courses). These students also cited positive competition 
among students as a factor that motivated them to succeed academically. They considered the 
school’s culture of making sure that all students understand the academic material to be a posi- 
tive feature of their SEED experience. 
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Student Life 

Perhaps the most distinctive aspect of SEED’S learning environment is the time that adults 
spend with students after school and through the evening. The Student Life department is re- 
sponsible for this time, developing and coordinating residential life programming and managing 
students’ time outside of the traditional academic day. 

Led by a director, the department includes middle management in the form of Student 
Life coordinators, who oversee and supervise life skills counselors (LSCs), responsible for de- 
veloping Student Life programming, and resident advisers (RAs), responsible for supervising 
students in the dormitories. Within the donns, students are organized into houses (or groups), 
each of which is led by an RA. Each house is named after a college or university and decorated 
with that institution’s memorabilia (pennants, pillows, and so on, prominently displayed in the 
house’s common spaces). LSCs work with the same grade level each year, while RAs remain 
with the same group of students throughout the students’ time at SEED. 

The Student Life department aims to develop students’ behavioral, social, and life skills 
while also reinforcing what they are learning in the classroom. In line with the school’s overall 
grade-specific cohort approach, there are specific goals for middle school, ninth-grade, and high 
school students. Middle school programming is intended to develop and refine social skills con- 
nected to meeting behavior expectations and routines, such as following instructions the first 
time they are given, adhering to the school dress code, and learning how to disagree appropri- 
ately. Ninth-grade Student Life activities aim to develop and reinforce the skills and habits nec- 
essary for success in high school, such as planning ahead, using anger control strategies, and 
building strong self-esteem. High school programming focuses on the transition to college, in- 
cluding preparation for the SAT, ACT, and Accuplacer entrance exams. 2 To accomplish these 
goals, the majority of Student Life time is structured, especially for the middle school grades. 
Figure 2.2 presents an example of a Student Life schedule, showing how sixth-graders spend 
their time outside of the school day. The following sections describe the wide variety of Student 
Life activities available to students. 

Students described the social and life skills they were being taught as an important step 
in their preparation for college and beyond. Students told of instructional and noninstructional 
staff members routinely discussing the personal habits and skills that students need to succeed 
in college, such as self-motivation, discipline, independence, strong time management skills (a 
recurring theme in all interviews and focus groups), leadership qualities, and other personal 
characteristics. Middle school students characterized their schedules as being overly reliant on 


2 The Accuplacer is a tool used by many colleges to assess students’ math, reading, and writing knowledge. 
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Figure 2.2 

Sample Student Life Schedule: Sixth Grade 


Time 

Monday 

Tuesday 

Wednesday Thursday 

5:30 am 

Wake up, make bed, 
prepare for the day 

Wake up, make bed, 
room inspection 

Wake up, make bed, prepare for the day 

6:00 am 

Uniform and room 
inspection 

Sparks (athletics, clubs, 

Uniform and room inspection 

6:15 am 

Breakfast 

etc.) 

Breakfast 

6:45 am 

Morning house meeting 

Breakfast and uniform 
inspection 

Morning house meeting 

7:45 am 

Resident adviser (RA) checks backpacks and school materials 

7:55 am 

Transition to school 

Academic school day 

3:30 pm 

Study Zone (Study Hall) 

5:00 pm 

Drop Everything and Learn (specified academic 
interventions based on needs) 

Prepare for dinner 

5:30 pm 

Dinner 

6:15 pm 

HALLS lesson (life skills 
education) 

Sparks (athletics, clubs, 
etc.) 

Character Education Sparks (athletics, clubs, 

lesson etc.) 

7:05 pm 

Drop Everything and Read (independent reading time) 

7:45 pm 

House programming (team-building, educational activities, games, etc.) 

8:30 pm 

Quiet house, showers, preparation for the next day 

9:15 pm 

Lights out and in bed 


SOURCE: SEED DC. 
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routines and related an overall sense of being overscheduled. Yet many reported taking ad- 
vantage of the various extracurricular activities offered at SEED. For example, some participat- 
ed on the track team and some in student government, and a few served as school ambassadors. 3 

Behavioral, Social, and Life Skill Development 

Student Life staff members described social skill development and life skills education 
as critical features of the SEED model. Students primarily engaged with these topics during 
SEED’S signature HALLS program, Character Education (at the middle school level), House 
Programming, and other grade-specific activities. 

The SEED-created Habits for Achieving Life-Long Success (HALLS) program teaches 
students social and basic life skills, such as decision making and communication strategies, and 
the importance of taking responsibility for oneself and others. HALLS activities focus on a va- 
riety of topics, such as bullying, dating relationships, and appropriate dining etiquette. During 
Character Education, middle school students leam about positive character development 
through activities such as watching movies and discussing various decisions and situations 
faced by the films’ characters. Box 2.1 describes HALLS and Character Education lessons that 
were observed during a site visit. House Programming provides time for discussions about 
weekly expectations, team-building activities, house celebrations, and participation in field trips. 
SEED DC implements other grade-specific activities when Student Life staff members deem 
necessary — for example, the Success Highways curriculum for eighth-grade students, a pro- 
gram that focuses on building resiliency, and the Search Institute’s I-Time social and emotional 
development curriculum for ninth-graders. 4 

Student Life staff members reported that they often chose topics for activities such as 
HALLS and Character Education based on their own assessment of students’ needs. They re- 
ported using established curricula and activities, developing their own lesson plans, or referring 
to lesson plans developed by staff in the past. Based on interviews with staff members and a few 
observations of Student Life activities, it was not clear to the evaluation team how student needs 
were identified, what specific outcomes Student Life programming was intended to affect, or 
how specific programming choices were expected to lead to desired outcomes. However, an 
administrator indicated that the school was planning to begin collecting annual survey data 
about student behavior and attitudes to understand areas where students may be demonstrating 
growth and where additional support is needed. 


3 School ambassadors are students who are chosen to represent the school, for example, by giving tours to 
campus visitors. 

4 For information on these curricula, see ScholarCentric (2015) and Search Institute (2016). 
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Box 2.1 

Sample Character Education and HALLS Lessons 
Character Education Lesson for Sixth-Grade Girls 

The 12 sixth-grade girls of Oxford House gathered for their weekly Character Education 
lesson. They sat in a circle in the common area, a bright and comfortable space decorated 
with colorful pictures and posters of the girls’ choosing and furnished with sofas and 
chairs. It took a few minutes for the girls to settle, but once they did, the resident adviser 
(RA) explained that the evening’s lesson would focus on expressing one’s opinion in a 
respectful manner. Then she passed around a hat containing topics written on small piec- 
es of paper. Each girl selected a topic from the hat and was tasked with talking about the 
issue for a full minute. The topics were wide-ranging, including abuse, teenage pregnan- 
cy, basketball, homework, and rap music, among others. One girl, sitting at a desk, served 
as the timer. Some girls struggled to speak for the full 60 seconds, but the RA encouraged 
them to try. After each girl had finished her one-minute speech, the RA invited others to 
share their opinions about the topic as well. Many of the girls challenged their peers and 
expressed different opinions. The RA used these opportunities to correct behaviors such 
as interrupting others and devaluing their perspectives and to model respectful disagree- 
ment and listening skill s . 

HALLS Lesson for Ninth-Graders 

Later in the evening, 12 ninth-grade girls gathered with their RA and their life skills 
counselor (LSC) for a HALLS lesson. Soon after gathering, the girls became quiet and 
listened for instructions. The RA asked the girls to reflect on a film they had watched and 
to consider one positive and one negative leadership quality that had been depicted in the 
film. The RA then distributed two Post-It notes to each girl and instructed them to record 
the negative and positive qualities on the notes and place them under the appropriate 
sheets that had been taped to the wall. After all the notes had been posted, the RA asked 
each girl to state a quality she had identified and to explain why it was either positive or 
negative. Halfway through the exercise, a student proposed a new activity. She asked the 
group to reflect on a different, more personal question: Which student in this house do 
you look up to and why? Many students acknowledged their admiration for girls in the 
house who demonstrated academic focus, were encouraging of others, or offered help to 
others with challenging homework assignments. 


Academic Supports and Activities 

Student Life time accommodates a set of academic supports and activities designed to 
help students excel. Each day all students must participate in Drop Everything and Read 
(DEAR), a 45-minute period reserved for independent and silent reading. In addition, eligible 
middle school students participate in Drop Everything and Learn (DEAL) twice per week, an 
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activity during which individually tailored interventions are used to address educational gaps in 
a variety of subjects. The school also offers students support through the SEED Tutoring En- 
richment Program. The tutoring program is led by the evening academic intervention coordina- 
tor and is staffed by SEED-trained volunteer tutors and SEED students who serve as peer tutors. 

Youth Development Activities and Recreational Time 

As a part of the Student Life curriculum, twice per week students are required to partic- 
ipate in a “spark,” an extracurricular activity or club (such as intramural volleyball, chess, yoga, 
or cosmetology) designed to help students explore their interests and develop motivation and 
initiative. 5 Besides helping students discover, nurture, and demonstrate their skills, sparks are 
designed to allow students to build healthy relationships with the adults who lead the activities. 
As explained below, athletic teams are a unit of Student Support Services, but students partici- 
pate in these activities before and after school during Student Life time. 


Student Support Services 

As a unit, Student Support Services is responsible for meeting the mental health, physical 
health, and social and emotional needs of students and addressing barriers that might prevent 
students from achieving their full potential. This department is led by a director who oversees 
five smaller departments: Mental Health, Health and Wellness, Athletics, External Opportuni- 
ties, and Dean of Students. These smaller units function as a combined approach to support stu- 
dents outside the classroom. 

SEED DC provides mental and physical health services that strongly resemble those at 
traditional schools, but because of the boarding setting, the staff must be available to provide 
assistance 24 hours a day. The Mental Health department is staffed by a director, two special 
education counselors, and two general education counselors who provide counseling services at 
the individual and group levels. The team offers one-on-one counseling for students who re- 
quest it and holds small group discussions for students on topics such as dealing with loss or 
body image issues. 

The Health and Wellness department consists of two nurses who address students’ 
medical needs, administer medicine, and lead workshops with Student Life staff members on 
topics such as personal hygiene, community living, and sex education. 

The director of Athletics manages the school’s athletic programming and the interscho- 
lastic sports program. Physical education and athletics include soccer, volleyball, golf, lacrosse, 


5 SEED worked directly with the Search Institute and used its research when designing this part of the Stu- 
dent Life program. 


18 



flag football, basketball, cheerleading, baseball, and track. Some athletic activities are held on 
campus; others, such as track, are held in offsite facilities. Through SEED DC’s membership in 
an independent school league, students are able to compete with students from other schools. 

SEED DC administrators described exposure to positive educational experiences and 
enrichment opportunities as a critical part of their model, citing the importance of developing 
well-rounded students who have the social skills to operate outside the SEED environment. 
With this goal in mind, Student Support Services includes a small External Opportunities de- 
partment whose coordinator is charged with actively increasing this exposure. Opportunities can 
take the form of students working with a local nonprofit organization to leam about efforts to 
stem the spread of HIV, or participating in international travel. To encourage students to make 
the most of the school’s external opportunities, all high school students must complete a sum- 
mer opportunity and all seniors must complete a semester-long internship before graduating. In 
focus groups, several students said they had wanted to attend SEED DC specifically because of 
the external opportunities offered. Some students who were initially lukewarm about enrolling 
in a boarding school reported that the opportunity to travel abroad had convinced them to enroll. 

SEED DC uses a skills-based approach to behavior management and discipline, believ- 
ing that students must develop various social skills in order to meet behavior expectations. 
Building on a Boys Town curriculum, SEED created the “Model of Care” as a school-wide sys- 
tem for this approach to skill development and behavior management. Though all staff mem- 
bers are responsible for implementing this system, the Dean of Students department is charged 
with overseeing its implementation. When students are unable to meet expectations, the dean 
and assistant deans are also involved in mediations and decision making about suspensions and 
expulsions. The Model of Care is discussed further below. 


College Counseling 

SEED students are expected to attend college following high school graduation. The College 
Counseling department, led by a director, is responsible for supporting students in the college 
search, application, and selection process. Early college awareness begins in the sixth grade, 
with students being encouraged to visit the College Cafe (the College Counseling department), 
a colorful and inviting space that houses informational resources and is decorated with memo- 
rabilia from many of the nation’s colleges. In middle school, students are engaged in discus- 
sions about the value of enrolling in college and participate in a college campus visit as well as 
activities to help develop academic habits that will prepare them for postsecondary success. 
Starting in ninth grade, students practice taking college entrance exams (the PSAT) so that they 
become familiar with the test and can improve their scores. High school students have access to 
ACT, SAT, and Accuplacer preparation materials during after-school hours. Students in elev- 
enth grade enroll in Junior College Seminar, in which advisers help them find the “right-fit” 
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college — the college that is the best academic match and that also meets their financial, social, 
and personal needs. In Senior College Seminar, students in the twelfth grade are actively en- 
gaged in the college search, application, and choice process. 

A distinctive feature of the SEED student experience is the support that SEED students 
and graduates receive from the College Transition and Success (CTS) team, a unit within the 
SEED Foundation. Working in collaboration with the College Counseling department, the CTS 
team holds a series of college transition workshops for seniors and their parents (including a 
financial literacy workshop) and helps students finish required college enrollment paperwork. 
The CTS team also monitors and supports SEED graduates as they begin college and maintains 
contact with them while they are enrolled in college, functioning as an important bridge be- 
tween SEED graduates’ high school and college experiences. The CTS team also perfonns an 
important data collection function. It collects and tracks college enrollment, withdrawal, and 
graduation data for SEED graduates, allowing the foundation to assess its graduates’ postsec- 
ondary progress . 6 

During focus groups, high school seniors said they felt supported by the College Coun- 
seling staff. They named staff members in the College Cafe and other “senior supporters” (staff 
members assigned to seniors to support them in other ways through their final year at SEED) as 
providing a great deal of encouragement during the college search and application process. 

In addition to preparing students for college, the department is charged with developing 
success plans for students who opt not to enroll in college after high school. Depending on the 
case, such postsecondary planning may include helping students prepare for a “gap year” or find 
employment that is aligned to their skill set. 


A Network of Supportive Adults and Peers 

SEED DC’s five-day-a-week residential model is designed to provide a supportive and integrat- 
ed learning environment for students. Therefore, it is essential that the four core programming 
departments coordinate, both in ways that students can directly see and experience and in more 
behind-the-scenes ways related to student needs and progress. 

School-Wide Systems and Climate 

To create consistency and provide an integrated learning environment, SEED uses a va- 
riety of school-wide systems that span the four programmatic departments. These practices in- 


6 An analysis of postsecondary retention and graduation data led the CTS Team to develop the “right fit” 
college criteria that the college counseling office uses in the advising process. 
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elude the previously mentioned Model of Care, a set of core values, and school-wide projects 
such as Presentations of Learning, discussed below. 

The Model of Care is at its simplest a behavior management system, overseen by SEED 
DC’s dean of students, but it plays a significant role in defining SEED culture and climate. All 
adults at SEED are expected to provide consistent messaging around behavior and expectations 
using this model. As previously described, the Model of Care defines a set of age-appropriate 
social skills that all students should possess. Staff members frequently name these skills when 
redirecting or praising students, especially in the middle school grades. Middle school students 
even carry around a “SEED note” which serves as a record of their behavior throughout each 
day. They receive “skills strengths” when exhibiting social skills defined in the Model of Care, 
and “improvement areas” when they misbehave. Points are awarded for skills strengths, and 
such points can be exchanged for prizes and other rewards. 

SEED DC also uses five core values to guide everything that happens at the school, as 
well as to set priorities for students. These values — responsibility, respect, self-discipline, 
compassion, and integrity — are displayed prominently throughout the school and are integrat- 
ed in annual student presentations called “Presentations of Learning,” or POLs. During the 
POL, a student gives a 30-minute presentation showcasing how he or she has applied the core 
values in academic and Student Life work and how he or she has grown over the year. The 
evaluation team observed two middle school student presentations. Unfortunately, neither of the 
students who were observed received high scores from the teachers who graded them. It was not 
clear to the evaluation team how much support the students had been given in preparation for 
their presentation. 

Communication and Relationships Among the Staff and Students 

With about 40 faculty and staff members in the academic department and about 50 staff 
members in the Student Life department, each student has access to a large network of adults 
when needed. Administrators, teachers, and students report that through fonnal and infonnal 
structures adults are actively involved in SEED students’ lives. For example, as previously men- 
tioned, both academic and Student Life staff members review interim assessment data to keep 
track of student progress and brainstonn ways to better support them. This monitoring means 
that the adults who are responsible for supervising students in the evening are intimately in- 
volved in their academic progress. In addition, teachers e-mail middle school and ninth-grade 
students’ homework assignments to their resident advisers, along with updates on student be- 
havior. In the morning, Student Life staff members brief the faculty about issues that may have 
surfaced in the donns overnight. Student Life staff members even sometimes sit in on classes so 
that they can understand what the students are learning and be better prepared to assist them. 
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Faculty and staff members agreed that ongoing communication among the staff is criti- 
cally important in supporting students’ success — and this communication appears to make it 
very difficult for students to “get lost” at the school. Student Life staff members reported that 
among students who shoulder adult responsibilities and pressures at home, the school is a safe 
haven; not only do students have access to nurturing and positive adult role models and cham- 
pions focused on helping them succeed, but according to an RA, at SEED DC students can feel 
“Em free to be me — a child.” 

Weekly Transitions to and from SEED 

Unlike at traditional boarding schools, SEED DC students leave campus every Friday 
afternoon and return on Sunday evening. Faculty and Student Life staff members reported that 
the weekend transition could be smooth for some students but challenging for others. For ex- 
ample, middle school students who may not have an enforced bedtime at home may struggle to 
adhere to the school’s bedtime curfew when they return to campus. Students who are not en- 
couraged to do their homework at home or do not have someone there to help them may return 
to campus without having completed assigmnents. To help students ease back into the SEED 
DC structure when they return, some RAs hold a Sunday evening check-in to leam about what 
may have transpired over the weekend. Staff members reported that these sessions allow stu- 
dents to “vent” and offer useful infonnation that often explains their behavior. These sessions 
also give staff members an opportunity to find ways to help students who may need support to 
meet SEED expectations when they return to campus. A Student Life staff member stated, 
“[Yjou’re able to bridge that gap. You’re able to find out okay, well, this weekend they went 
through X, Y, and Z so they’re having a difficult day so now what can we do? You’re able to be 
more solution-oriented.” While the transition can be tough for some students, Student Life staff 
members explained that students view their peers as family, and when they return to campus 
they are often excited about returning and are eager to share updates about their weekend expe- 
rience. 


In focus groups, student participants explained that the transition could be challenging 
in other ways. All students reported that they do not have enough time on the weekends to 
spend it as they like, and a few mentioned not having enough time to devote to their relation- 
ships at home. Two ninth-grade students explained that after spending so many years at SEED 
DC, they noticed that they had “separated” or become distant from family because they were 
not home during the week to interact with their relatives . 7 High school students also cited dif- 
ferent school schedules as a factor preventing them from spending as much time with friends at 
home as they would like. 


7 One student suggested that the school hold a workshop to help students handle these changes in their per- 
sonal relationships. 
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Chapter 3 

The Academic and Behavioral Impacts of SEED 


This chapter focuses on the academic and behavioral impacts of SEED on its students. It begins 
by briefly describing the study’s random assigmnent process and the characteristics of the study 
sample as a whole and explains the construction of the three analytic samples used in this re- 
search. The chapter then describes the short- and long-term impacts of SEED on students’ 
school experiences and academic and behavioral outcomes. SEED’S effectiveness is primarily 
measured using students’ scores on state reading and math exams and four-year high school 
graduation rates. Students’ survey responses regarding their school experiences and behaviors 
are secondary outcomes in this evaluation and are presented as a means of putting the pattern of 
academic effects in context. 


Random Assignment and the Study Sample 

As mentioned in Chapter 1 , the research team took advantage of the public SEED charter school 
admissions process to identify eight cohorts of students who had competed in a lottery for the 
opportunity to attend SEED. 1 The students who won the lottery are known in experimental re- 
search as the “treatment group” and are referred to in this text as the SEED group; the students 
who lost the lottery are known as the “control group,” or the non-SEED group. Because a lot- 
tery is a form of random assigmnent, the two groups should be comparable, and therefore any 
difference in study outcomes between the two groups, on average, can be attributed to SEED. 
The research team’s analytic approach is described in greater detail in Appendix D. 

Table 3.1 indicates that students in the study sample were primarily African-American 
and were economically and academically disadvantaged. Of the SEED group, roughly 46 per- 
cent of the students had direct certification status, meaning they automatically qualified for a 
free lunch because they were in foster care, were homeless, or were living in a household that 
received income or nutritional assistance from certain government programs. 2 An additional 33 
percent of the SEED group students qualified for free or reduced-price lunch on the basis of 
household income level; in sum, 79 percent of the students qualified for free or reduced-price 
lunch. In terms of their academic characteristics, in the year they applied to SEED 14 percent of 
the SEED group students qualified for special education services, 46 percent scored at or above 


'The SEED admissions process is described in greater detail in Appendix C. 

2 Benefits that qualify a household for direct certification are the Supplemental Nutrition Assistance Pro- 
gram (SNAP), the Food Distribution Program on Indian Reservations (FDPIR), and Temporary Assistance for 
Needy Families (TANF). 
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SEED Evaluation Final Report 
Table 3.1 

Baseline Characteristics of SEED Lottery Participants: Full Sample 


Characteristic 

SEED 

Group 

Non-SEED Estimated 

Group Difference 

P-Value for 

Estimated 

Difference 

Race (%) 





Black 

98.8 

98.4 

0.4 

0.695 

Female (%) 

49.1 

49.1 

0.0 

1.000 

Economic indicator (%) 





Eligible for free/reduced-price lunch 

78.5 

87.4 

_g 9 ** 

0.017 

Direct Certification status 2 

45.9 

45.1 

0.8 

0.891 

Special education status (%) 

13.6 

16.9 

-3.3 

0.360 

Standardized math score 

2.1 

-11.0 

13.1 

0.194 

At or above proficiency level in math (%) 

46.1 

39.4 

6.7 

0.287 

Standardized reading score 

6.6 

-8.8 

15.4 

0.133 

At or above proficiency level in reading (%) 

47.0 

40.7 

6.4 

0.309 

Sample size 

499 

267 




SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) 
student enrollment data and state test scores from the 2006-2007 to 2011-2012 school years. 

NOTES: Values for SEED group members are the simple means or percentages. Values for the 
difference between SEED group members and non-SEED group members are obtained from a 
regression of a given baseline characteristic on a series of indicator variables that identify each lottery, 
a covariate indicating the probability that a student would win a spot on lottery day, and an indicator 
variable that equals 1 for lottery winners and 0 for lottery losers. The coefficient on the latter 
indicator variable equals the difference in the baseline characteristic between SEED group members 
and non-SEED group members. The value for non-SEED group members equals the corresponding 
value for SEED group members minus the estimated difference between SEED group members and 
non-SEED group members. 

Data on students' income, proficiency in math, and proficiency in reading are not available for 
students in Cohorts 1 and 2. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 
indicated as: ** = 5 percent; * = 10 percent. 

“Direct Certification is automatic qualification for free lunch because the individual is in foster 
care, is homeless, or is living in a household receiving benefits from the Supplemental Nutrition 
Assistance Program (SNAP), the Food Distribution Program on Indian Reservations (FDPIR), or 
Temporary Assistance for Needy Families (TANF). 
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proficient on the state math exam, and 47 percent scored at or above proficient on the state read- 
ing exam. A comparison with the non-SEED group characteristics shows that the SEED group 
students were 8.9 percentage points less likely to have been eligible for free or reduced-price 
lunch (a statistically significant difference) and roughly 6 percentage points more likely to have 
scored at or above proficiency on their reading and math exams. Although random assignment 
is expected to produce two groups that are equivalent on all measurable characteristics, with 
small samples it is common to see differences such as these. 3 Appendix D discusses this issue in 
greater detail. 

At baseline the academic characteristics of the SEED study sample appear similar to 
those of all students in District of Columbia Public Schools (DCPS), but the SEED study sam- 
ple is slightly more economically disadvantaged. During the study period, on average, 75 per- 
cent of the students in DCPS were African-American and 43 percent of the students in DCPS 
scored at or above proficient on their sixth-grade math exam, compared with 46 percent of the 
students in the SEED study sample. While sixth-grade-specific data on DCPS students’ family 
income levels are unavailable, during this time period roughly 7 1 percent of all the students in 
DCPS were eligible for free or reduced-price lunch, compared with 79 percent of the SEED 
study sample. It is important to note that in other cities SEED strives to work with an even 
greater proportion of students who are economically disadvantaged and gives these students 
preference during the school enrollment process. SEED DC is unable to do so given the District 
of Columbia’s strict charter school enrollment guidelines. 


The Three Analytic Samples 

Originally students entered SEED as seventh-graders, but in the spring of 2009 SEED changed 
its model and began to enroll sixth-graders as well. For the next two years, students could enter 
SEED as either sixth- or seventh-graders. Then, in spring 2011, SEED fully transitioned to its 
new model and admitted students only as sixth-graders. As a result, the study sample consists of 
eight cohorts of students who competed in a lottery to attend SEED across six school years. Co- 
horts 1-4 contain students who competed in a lottery to begin SEED as seventh-graders in 2006- 
2007 through 2009-2010. Together, these four cohorts are referred to as the Seventh-Grade En- 
trant Sample. Cohorts 5-8 contain students who competed in a lottery to begin SEED as sixth- 
graders from 2008-2009 through 2011-2012. These cohorts are referred to as the Sixth-Grade 
Entrant Sample. The third analytic sample, consisting of cohorts 1 and 2, is described below. 

Students in the Seventh-Grade Entrant Sample were analyzed separately from those in 
the Sixth-Grade Entrant Sample for a few reasons. First, since traditional middle schools begin 


3 ln accordance with What Works Clearinghouse (2014) recommendations, these characteristics are in- 
cluded as co variates in the impact model. 
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in sixth grade, students who wanted to leave their middle school and enter SEED as seventh- 
graders may be a self-selected sample of students who were particularly dissatisfied with their 
current middle school. Second, as students are changing rapidly during early adolescence, there 
may have been developmental differences between those who entered SEED as sixth-graders 
and those who entered as seventh-graders, and those differences may have affected how they 
responded to SEED’S unique environment and programming. Finally, since SEED changed its 
model when it began admitting sixth-graders, the two different samples may have experienced 
slightly different versions of the SEED program. 

The research team collected school records data from the District of Columbia’s Office 
of the State Superintendent of Education (OSSE). 4 These data consisted of statewide student 
enrollment records, state exam performance data (each year the District of Columbia adminis- 
ters statewide reading and math exams to sixth-, seventh-, eighth-, and tenth-graders), student 
demographic data, and information on whether and when a student graduated from high school. 
Box 3.1 offers more information on these and other outcomes included in our analysis. These 
data were available for all DC students attending public schools — District of Columbia Public 
Schools and charter schools. The team collected these data for the 2006-2007 school year 
through the 2013-2014 school year. 

Table 3.2 shows the OSSE school records data follow-up period, by cohort, for the Sev- 
enth-Grade and Sixth-Grade Entrant Samples; the years in which students took the OSSE state 
exam are in bold. Three years of OSSE test score data are available for all four cohorts of the 
Seventh-Grade Entrant Sample and for three of the four cohorts in the Sixth-Grade Entrant 
Sample. The research team was able to follow a subsample of the Seventh-Grade Entrant Sam- 
ple, students in cohorts 1 and 2, for six years, through their anticipated four-year high school 
graduation; this group is referred to as the Cohorts 1 and 2 Sample. Analyses in this chapter 
draw on the full Seventh- and Sixth-Grade Entrant Samples to estimate the “short-term” effects 
of SEED and the Cohorts 1 and 2 Sample to estimate the “long-term” effects of SEED. 5 Be- 
cause of their different sizes, these samples have differing levels of statistical power (the ability 
to detect a true impact with statistical significance). Specifically, the full Seventh- and Sixth- 
Grade Entrant Samples are larger, enabling the study to detect impacts of roughly 0.14 standard 
deviation at a 95 percent probability level, while the Cohorts 1 and 2 Sample requires an impact 
of 12.7 percentage points on four-year high school graduation to show statistical significance. 
Although these minimum detectable effects would be characterized as moderate to large by ed- 
ucation researchers, they are, arguably, not unreasonable for such an intensive intervention. Be- 
cause of these already small sample sizes we are unable to explore whether the effects of SEED 
differ across student subgroups. 

4 “State” refers to the District of Columbia. 

5 The baseline characteristics of the Cohorts 1 and 2 Sample are presented in Appendix D; they appear sim- 
ilar to those of the full sample presented in Table 3.1. 
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Box 3.1 

Definitions of Study Outcomes 

Academic Achievement Outcomes 

• Standardized DC CAS scaled scores. During the evaluation period, the District of 
Columbia used the DC Comprehensive Assessment System (DC CAS) to measure 
students’ mastery of the DC Content Standards. Students took the DC CAS math test 
in grades 3 through 8 and grade 10, and reading tests were administered in grades 2 
through 10. (In grades 2 and 9, scores were reported only internally.) Students in 
grades 5 and 8, as well as those enrolled in Biology, also took the science test. Stu- 
dents in grades 4, 7, and 10 took the composition test. Finally, the health test was ad- 
ministered in grades 5 and 8 and during the high school year in which students take 
health class. This outcome measure reports student performance using their standard- 
ized scaled scores. 

• At or above DC CAS proficiency level. Student performance on the DC CAS is also 
reported using proficiency levels determined by standardized scaled scores: advanced 
(exceeding state standard), proficient (meeting state standard), basic (not meeting state 
standard), and below basic (not meeting state standard). 

• Four-year high school graduation. This measure indicates whether a student gradu- 
ated from high school within four years. 

Student Experience Outcomes 

• Courses taken. The outcome includes the number of Advanced Placement or honors 
courses students took, as well as an indicator of whether they took other important 
high school courses, such as Calculus and Statistics. 

• Extracurricular activities. Students were asked whether they participated in athletics, 
clubs, and so on, during the 2012-2013 school year or the last year that they attended 
school. 

• Academic support. Students were asked whether they received support such as tutor- 
ing and personalized academic counseling during the 2012-2013 school year or the 
last year that they attended school. 

• College preparatory activities. Students were asked whether they participated in ac- 
tivities to prepare them for college, such as visiting college campuses and learning 
about the admissions process, during the 2012-2013 school year or the last year that 
they attended school. 

• Work-readiness activities. Students were asked whether they participated in activities 
related to career exploration or work readiness, such as working as an intern or shad- 
owing someone on a job, during the 2012-2013 school year or the last year that they 
attended school. 

('continued') 
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Box 3.1 (continued) 

• School atmosphere. Students were asked about their perceptions of their school dur- 
ing the 2012-2013 school year or the last year they attended school — for example, 
whether they had orderly classrooms and whether they had a caring adult at school. 

• Living environment during the week. Students were asked about their perceptions 
of the place that they lived during the week in the 2012-2013 school year or the last 
year that they attended school. For example, they were asked whether they felt that 
they had adult support during the week and whether they ate three meals per day. 

Behavioral Outcomes 

• School behaviors. Students were asked about certain behaviors related to academic 
success, including study habits and organizational skills. 

• Time use. Students were asked how many hours they spent doing various activities 
during the week, such as homework, reading, and video games. 

• Likelihood of attending college. Students were asked how likely they were to attend 
college. 

• Risky behaviors. Students were asked how many times they drank alcohol or skipped 
school, as well as their overall engagement in risky behaviors — such as having un- 
protected sex or getting involved with the criminal justice system — in the years since 
random assignment. 

• Metacognitive skills. Students’ responses to questions about their personal character- 
istics and outlook were scored on scales used to measure “grit” and self-control. 


Short-Term Effects of SEED 

Difference in Student Experience 

The District of Columbia is an especially charter-rich environment, and many of the 
students who applied to SEED but did not win the lottery actively sought out other innovative 
school options. Specifically, of the SEED lottery losers, 43 percent of the students enrolled in 
charter schools and a little more than half enrolled in traditional DC middle schools. Thus in this 
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Table 3.2 


Follow-Up Periods by Cohort 


Cohort 

Lottery Y ear 

2006-2007 

2007-2008 

2008-2009 

2009-2010 

2010-2011 

2011-2012 

2012-2013 

2013-2014 

7th-grade entrants 

1 2007 

Lottery year 
(6th grade) 

year 1 
(7th grade) 

year 2 
(8th grade) 

year 3 
(9th grade) 

year 4 
(10th grade) 

year 5 

(11th grade) 

year 6 
(12th grade) 


2 

2008 


Lottery year 
(6th grade) 

year 1 
(7th grade) 

year 2 
(8th grade) 

year 3 
(9th grade) 

year 4 
(10th grade) 

year 5 

(11th grade) 

year 6 

(12th grade) 

3 

2009 



Lottery year 
(6th grade) 

year 1 
(7th grade) 

year 2 
(8th grade) 

year 3 
(9th grade) 

year 4 
(10th grade) 

year 5 

(11th grade) 

4 

2010 




Lottery year 
(6th grade) 

year 1 
(7th grade) 

year 2 
(8th grade) 

year 3 
(9th grade) 

year 4 
(10th grade) 

6th-grade entrants 

5 2009 



Lottery year 
(5th grade) 

year 1 
(6th grade) 

year 2 
(7th grade) 

year 3 
(8th grade) 

year 4 
(9th grade) 

year 5 
(10th grade) 

6 

2010 




Lottery year 
(5th grade) 

year 1 
(6th grade) 

year 2 
(7th grade) 

year 3 
(8th grade) 

year 4 
(9th grade) 

7 

2011 





Lottery year 
(5th grade) 

year 1 
(6th grade) 

year 2 
(7th grade) 

year 3 
(8th grade) 

8 

2012 






Lottery year 
(5th grade) 

year 1 
(6th grade) 

year 2 
(7th grade) 


NOTE: The follow-up periods that coincide with years in which students took the Office of the State Superintendent of Education (OSSE) 
state exam are in bold. 



study the SEED student experience was compared with a diverse set of other school experiences 
that may not exist in other contexts. 

Not all the students who won the SEED lottery enrolled in SEED the following fall. 
Specifically, 18 percent of the Seventh-Grade Entrant Sample’s lottery winners and 21 percent 
of the Sixth-Grade Entrant Sample’s lottery winners did not enroll in SEED. Sixty-three percent 
of these students enrolled in other charter schools, and the remainder enrolled in traditional DC 
middle schools. Even though some of the SEED lottery winners did not enroll in SEED, this 
chapter presents findings from an analysis of the difference in outcomes between all the SEED 
lottery winners and all the SEED lottery losers. Known as an “intent-to-treat” approach, this 
analysis is described as the impact of being offered the opportunity to enroll in SEED. Appen- 
dix E presents findings from an exploratory analysis of the effect of enrolling in SEED, known 
as “treatment on the treated.” Generally, estimates of the effect of enrolling in SEED are about 
15 percent to 20 percent larger in magnitude than the estimated intent-to-treat effects reported in 
the remainder of this chapter, but the overall pattern of effects and statistical significance is the 
same. 


Effects on Academic Achievement 

In each follow-up year, students’ academic achievement was measured by their perfor- 
mance on the District of Columbia’s statewide reading and math exams. After the first follow- 
up year some students progressed into the next grade while others did not. In order to keep the 
analysis sample intact, the research team reports each student’s exam score as a z-score, which 
is standardized on the sample mean and standard deviation of the student’s grade level; profi- 
ciency levels are also based on the exams students actually took. (To be consistent over time, 
students’ scores in the first follow-up year are standardized in this manner as well.) The estimat- 
ed difference in standardized math scores is referred to as the “effect size.” 

Findings for the Seventh-Grade Entrant Sample in Table 3.3 indicate that on average, in 
the first two years of follow-up, being offered the opportunity to attend SEED increased stu- 
dents’ academic achievement in math. The first two rows in the table present findings from an 
analysis of students’ performance on the OSSE math exam in the first follow-up year. On aver- 
age, students in the SEED group scored 0.14 standard deviation above the sample mean, while 
students in the non-SEED group scored 0.1 1 standard deviation below the sample mean, result- 
ing in an estimated difference of 0.24 standard deviation. The typical annual gain for seventh- 
graders in math is 0.32 standard deviation. 6 Thus, the SEED effect of 0.24 standard deviation is 
roughly equivalent in magnitude to 75 percent of the typical annual gains for this age group. 
Said differently, at the end of the first follow-up year, students in the SEED group were roughly 


6 Hill, Bloom, Black, and Lipsey (2008). 
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Table 3.3 


Estimated Effects of Being Offered the Opportunity to Attend SEED, 
Seventh-Grade Entrant Sample 


Outcome 

SEED 

Group 

Non-SEED 

Group 

Estimated 

Difference 

P-Value for 

Estimated 

Difference 

First year 3 





Standardized math score 

0.14 

-0.11 

0.24 ** 

0.027 

At or above proficiency level in math (%) 

66.3 

48.8 

17.5 ** 

0.035 

Standardized reading score 

0.05 

0.04 

0.01 

0.920 

At or above proficiency level in reading (%) 

44.9 

49.8 

-4.9 

0.532 

Second year 





Standardized math score 

0.14 

-0.18 

0.33 ** 

0.012 

At or above proficiency level in math (%) 

67.4 

46.2 

21.2 *** 

0.001 

Standardized reading score 

0.08 

-0.15 

0.23 * 

0.059 

At or above proficiency level in reading (%) 

55.6 

37.9 

27 g *** 

0.007 

Sample size 

184 

165 




SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) state 
test scores from the 2007-2008 to 2011-2012 school years. 


NOTES: The Seventh-Grade Entrant Sample consists of 349 students who participated in a lottery for 
SEED in the spring of 2007, 2008, 2009, or 2010. 

Values for SEED group members are the simple means or percentages. Values for the difference 
between SEED group members and non-SEED group members are obtained from a regression of a 
given outcome on a series of indicator variables that identify each lottery, a student's baseline reading 
score, a student's baseline math score, indicators of family income, and an indicator variable that 
equals 1 for lottery winners and 0 for lottery losers. The coefficient on the latter indicator variable 
equals the difference in outcome between SEED group members and non-SEED group members. The 
value for non-SEED group members equals the corresponding value for SEED group members minus 
the estimated difference between SEED group members and non-SEED group members. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a First-year DC Comprehensive Assessment System (CAS) data are only available for students who 
participated in a lottery for SEED in the spring of 2008, 2009, or 2010. 
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three-quarters of a year ahead of students in the non-SEED group in math. According to 
OSSE’s proficiency benchmark, this increase in SEED group students’ math scores translated 
into an 1 8 percentage point increase in the proportion of students scoring at or above proficient. 
Both of these effects are statistically significant. 

While students in the SEED group did not perform better than students in the non-SEED 
group in their first year of follow-up on the OSSE reading exam, they performed higher on both 
their math and reading exams in the second follow-up year. The second panel in Table 3.3 indi- 
cates that in the second follow-up year, being offered the opportunity to attend SEED increased 
students’ test scores over those of the non-SEED group by the equivalent of one and a half years 
of typical growth in math and one year of typical growth in reading (effect sizes = 0.33 and 0.23, 
respectively). 7 According to the OSSE benchmarks, SEED group students were 21 percentage 
points more likely to be at or above proficient in math and 1 8 percentage points more likely to be 
at or above proficient in reading. Both of these effects are statistically significant. 

Findings in Table 3.4 indicate that students in the Sixth-Grade Entrant Sample experi- 
enced a similar, though slightly weaker, pattern of positive SEED effects. Specifically, similar to 
the seventh-grade entrants, in the first follow-up year students in the SEED group scored 0.18 
standard deviation higher on their math exam than students in the non-SEED group, an effect 
equivalent in magnitude to 60 percent of the typical annual gains for this age group. 8 According 
to OSSE’s proficiency benchmark, this increase in SEED group students’ math scores translated 
into an 19.6 percentage point increase in the proportion of students scoring at or above proficient. 
The effect on students’ standardized math scores does not quite meet this report’s threshold for 
statistical significance, but the effect on students’ proficiency levels is statistically significant. 

Like the seventh-grade entrants, SEED group students in the Sixth-Grade Entrant Sam- 
ple did not perform better than students in the non-SEED group in their first year of follow-up 
on the OSSE reading exam, but in the second follow-up year they performed higher on both 
their math and reading exams. The second panel in Table 3.4 indicates that in the second fol- 
low-up year, being offered the opportunity to attend SEED increased students’ test scores by the 
equivalent of roughly 1 .2 years of growth in math (effect size = 0.39) and 8 1 percent of the typ- 
ical gains for this age group in reading (effect size = 0.21). 9 Unlike the Seventh-Grade Entrant 
Sample findings, these test score gains did not translate into a large increase in the proportion of 
students passing the OSSE proficiency benchmark. SEED group students were only 6.3 per- 
centage points more likely to be at or above proficient in math and 4.6 percentage points more 
likely to be at or above proficient in reading. Neither of these effects is statistically significant. 


Hill, Bloom, Black, and Lipsey (2008). 
8 Hill, Bloom, Black, and Lipsey (2008). 
9 Hill, Bloom, Black, and Lipsey (2008). 
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Table 3.4 


Estimated Effects of Being Offered the Opportunity to Attend SEED, 
Sixth-Grade Entrant Sample 


Outcome 

SEED 

Group 

Non-SEED 

Group 

Estimated 

Difference 

P-Value for 

Estimated 

Difference 

First year 





Standardized math score 

0.06 

-0.11 

0.18 

0.116 

At or above proficiency level in math (%) 

49.8 

30.2 

jq g *H=H= 

0.002 

Standardized reading score 

0.06 

-0.01 

0.07 

0.539 

At or above proficiency level in reading (%) 

34.2 

34.2 

0.0 

0.998 

Second year 





Standardized math score 

0.13 

-0.27 

Q 3q H=H=H= 

0.001 

At or above proficiency level in math (%) 

63.4 

57.1 

6.3 

0.304 

Standardized reading score 

0.09 

-0.12 

0.21 * 

0.072 

At or above proficiency level in reading (%) 

48.6 

44.0 

4.6 

0.478 

Third year a 





Standardized math score 

0.06 

-0.13 

0.19 

0.181 

At or above proficiency level in math (%) 

70.6 

60.1 

10.5 

0.138 

Standardized reading score 

-0.02 

0.01 

-0.03 

0.789 

At or above proficiency level in reading (%) 

51.3 

55.3 

-4.0 

0.570 

Sample size 

315 

102 




SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) state 
test scores from the 2009-2010 to 2013-2014 school years. 


NOTES: The Sixth-Grade Entrant Sample consists of 417 students who participated in a lottery for 
SEED in the spring of 2009, 2010, 2011, or 2012. 

Values for SEED group members are the simple means or percentages. Values for the difference 
between SEED group members and non-SEED group members are obtained from a regression of a 
given outcome on a series of indicator variables that identify each lottery, a student's baseline reading 
score, a student's baseline math score, indicators of family income, and an indicator variable that 
equals 1 for lottery winners and 0 for lottery losers. The coefficient on the latter indicator variable 
equals the difference in outcome between SEED group members and non-SEED group members. The 
value for non-SEED group members equals the corresponding value for SEED group members minus 
the estimated difference between SEED group members and non-SEED group members. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a Third-year DC Comprehensive Assessment System (CAS) scores are only available for students 
who participated in a lottery for SEED in the spring of 2009, 2010, or 201 1. 
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Surprisingly, when the Sixth-Grade Entrant Sample is followed into its third year, the 
second year’s positive effects in math return to levels slightly lower than those seen in the first 
year and the positive effects in reading disappear. 10 The third panel in Table 3.4 indicates that in 
the third follow-up year, being offered the opportunity to attend SEED increased students’ test 
scores by the equivalent of roughly 86 percent of the typical gains for this age group (effect size 
= 0.19). 11 This increase in SEED group students’ math scores translated into a 10.5 percentage 
point increase in the proportion of students scoring at or above proficient. Neither of these ef- 
fects is statistically significant, though the p-value of the effect on the percentage of students 
scoring at or above proficient in math is just above this report’s 0.10 benchmark for signifi- 
cance. The effect of SEED on students’ reading scores in the third year is unclear. Although the 
estimated difference between the SEED group and the non-SEED group appears negative on 
both reading measures, there is a high probability that the apparent effects are due to chance. 
Thus, it is difficult to determine whether the SEED effect on student’s reading scores persists, 
disappears, or changes direction in students’ third year. 

To conclude, for students in both samples, being offered the opportunity to enroll in 
SEED led to a pattern of positive short-term effects. Students in each follow-up year experi- 
enced positive effects in math, while the effect on reading was most prominent in students’ sec- 
ond year. Although there is limited statistical power to explore any differences between cohorts, 
it appears that students in the Seventh-Grade Entrant Sample may have experienced slightly 
stronger SEED effects than students in the Sixth-Grade Entrant Sample. This may mean that 
SEED is more effective when students are offered the opportunity to enroll as seventh-graders, 
or it may mean that SEED became slightly less effective over time. 


Longer-Term Effects of SEED 

The research team was able to follow a subsample of the Seventh-Grade Entrant Sample, stu- 
dents in cohorts 1 and 2, for six years, through their anticipated four-year high school gradua- 
tion. When analyzed separately, students in these two cohorts experienced effects similar in 
magnitude and statistical significance to those experienced by all four cohorts of the Seventh- 
Grade Entrant Sample in their first two years in math. Students in cohorts 1 and 2 experienced a 
somewhat smaller effect in reading in their second year than did the full Seventh-Grade Entrant 
Sample. 


IH Only cohorts 5, 6, and 7 can be followed into a third year. 
"Hill, Bloom, Black, and Lipsey (2008). 
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Difference in Student Experience 

In their third follow-up year students in cohorts 1 and 2 transitioned into high school. 12 
As described in Chapter 2, SEED’S high school model is different from the middle school mod- 
el in a few key ways — boys and girls are taught in the same classrooms, students’ time after 
school is less structured, and students begin more rigorous college preparatory activities. Be- 
tween the second and third follow-up years, roughly 20 percent of the SEED lottery winners 
enrolled in SEED chose to leave and attend a different high school. Of this 20 percent, two out 
of five enrolled in a DCPS high school and the rest enrolled in another charter school. The stu- 
dents who chose to leave SEED at this point had, on average, seventh- and eighth-grade state 
test scores within 0.03 standard deviation of those who stayed at SEED, so there is no evidence 
that these students were academically struggling at SEED before their departure. As discussed 
in Appendix E, the proportion of SEED group students enrolled in SEED decreased over time. 
By twelfth grade, 44 percent of cohort 1 SEED group students and 63 percent of cohort 2 SEED 
group students were actually enrolled at the school. This issue is addressed further in Chapter 4. 

In the fall of 2013 the research team surveyed lottery winners and lottery losers in co- 
horts 1 and 2 and asked them a variety of questions about their experiences in school. Box 3.1 
provides more detail about the types of information captured by these questions and the relevant 
outcomes included in the analysis. At that point, most of the students in those cohorts were high 
school seniors or had recently graduated. Although the survey response rate was reasonably high 
(73 percent overall), the resulting small sample size decreases the study’s already limited statisti- 
cal power. (The construction and administration of this survey is described in Appendix F.) 
Overall, students in the SEED group reported experiencing a more rigorous and supportive aca- 
demic environment. Specifically, while only a fraction of the measures reveal statistically signifi- 
cant differences, Table 3.5 shows that students in the SEED group took fewer Advanced Place- 
ment courses but a larger number of foreign language, advanced science, and advanced math 
courses; participated in more extracurricular activities; received more academic support from 
teachers and adults; and participated in more college preparatory and work-readiness activities. 
In addition, SEED group students reported having more orderly classrooms and more academi- 
cally motivated peers. Students did not report a difference in weekday living environments. 

Effects on Academic Achievement 

Findings for cohorts 1 and 2 in Table 3.6 indicate that on average, in the fourth follow- 
up year (tenth grade for most students), being offered the opportunity to attend SEED appears to 
have increased students’ academic achievement in math but not in reading. Specifically, in the 
fourth follow-up year students in the SEED group scored 0.20 standard deviation higher on their 


' in the Cohorts 1 and 2 Sample, 10 percent of the students were retained in grade. 
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Table 3.5 

Estimated Differences in Student Experience, 
Cohorts 1 and 2 


Outcome 

SEED 

Group 

Non-SEED 

Group 

Estimated 

Difference 

P- Value for 
Estimated 
Difference 

Courses taken 

Number of Advanced Placement/honors courses 

1.50 

1.90 

-0.40 

0.252 

taken in high school (0-8) a 

Number of years of foreign language taken (0-4 ) h 

2.35 

2.01 

0.34 ** 

0.050 

Took physics in high school (%) 

72.2 

55.5 

16.7 * 

0.077 

Took calculus in high school (%) 

31.9 

20.9 

10.9 

0.204 

Took statistics in high school (%) 

44.0 

35.0 

8.9 

0.372 

Activities at school 

Extracurricular activities (l-5) c 

Went on field trips 

2.75 

2.18 

0.57 *** 

0.006 

Participated in sports/athletic teams at school 

3.15 

3.02 

0.13 

0.704 

Participated in arts, music group, literary clubs 

2.71 

2.60 

0.11 

0.730 

Participated in student leadership groups 

2.54 

1.98 

0.57 * 

0.074 

or student government 

Participated in community service activities 

3.28 

3.06 

0.22 

0.375 

Academic support 

Received tutoring/extra help from a teacher (l-5) c 

3.19 

2.86 

0.33 

0.205 

Received tutoring/extra help from another adult (l-5) c 

3.18 

2.59 

0.58 ** 

0.034 

Received personalized academic counseling (%) 

80.2 

78.0 

2.3 

0.784 

Received help preparing for DC-CAS exam (l-5) c 

2.73 

2.54 

0.18 

0.506 

College preparatory activities 

Score on access to college preparatory 
activities scale (l-3) d 

2.50 

2.49 

0.01 

0.936 

Received help preparing for ACT, SAT, PSAT, 
or other college entrance exams (l-5) c 

2.89 

2.79 

0.11 

0.672 

Went on college visits in the DC area (l-5) c 

2.53 

2.22 

0.32 

0.167 

Went on college visits in another state (l-5) c 

2.24 

2.01 

0.23 

0.242 

Work-readiness activities (l-5) c 

Had an internship 

2.35 

1.80 

0.56 ** 

0.040 

Participated in job shadowing or visits to 
workplaces 

1.84 

1.68 

0.16 

0.498 


(continued) 
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Table 3.5 (continued) 


Outcome 

SEED 

Group 

Non-SEED 

Group 

Estimated 

Difference 

P- Value for 
Estimated 
Difference 

School atmosphere (1-4) 

Score on school comfort scale d 

3.06 

3.11 

-0.04 

0.691 

Score on classroom order scale d 

2.83 

2.59 

0.25 * 

0.063 

Score on peers' academic motivation scale d 

3.37 

2.96 

0.41 *** 

0.003 

Score on caring adult at school scale d 

3.50 

3.46 

0.04 

0.716 

Score on school-wide future orientation scale d 

3.32 

3.31 

0.01 

0.929 

Weekday home livins environment 

Score on order of home during the week scale (l-4) d 

2.94 

3.18 

-0.24 

0.246 

Score on caregiver support during the week scale (l-4) d 

3.39 

3.40 

-0.01 

0.936 

Days in last week ate breakfast (0-7) e 

4.08 

3.76 

0.32 

0.510 

Days per week ate three meals per day (0-7) e 

4.51 

4.68 

-0.18 

0.704 

Days per week engaged in physical exercise (0-7) e 

4.45 

3.98 

0.46 

0.292 

Score on average weeknight sleep scale (l-7) f 

3.69 

3.91 

-0.22 

0.399 

Sample size 

121 

93 




SOURCE: MDRC calculations based on data collected in student survey. 


NOTES: Cohorts 1 and 2 consist of 214 students who were sixth-graders in the springs of 2007 and 
2008, respectively. 

Values for SEED group members are the simple means or percentages. Values for the difference 
between SEED group members and non-SEED group members are obtained from a regression of a 
given outcome on a series of indicator variables that identify each lottery, a student's baseline reading 
score, a student's baseline math score, indicators of family income, and an indicator variable that 
equals 1 for lottery winners and 0 for lottery losers. The coefficient on the latter indicator variable 
equals the difference in outcome between SEED group members and non-SEED group members. The 
value for non-SEED group members equals the corresponding value for SEED group members minus 
the estimated difference between SEED group members and non-SEED group members. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are indicated 
as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a This item is based on the following range: 0-8 courses. 
b This item is based on the following range: 0-4 years. 

c These items are based on the following scale: 1: Never; 2: Once a term/semester; 3: Monthly; 4: 
Weekly; 5: Almost every day. Therefore, a higher score indicates a higher frequency. 
d See Appendix F for definitions of the scales used in this report. 
e This item is based on the following range: 0-7 days. 

f This item is based on the following scale: 1: 4 hours or less; 2: 5 hours; 3: 6 hours; 4: 7 hours; 5: 8 
hours; 6: 9 hours; 7:10 hours or more. 
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Table 3.6 


Estimated Effects of Being Offered the Opportunity to Attend SEED on 
Academic Achievement, Cohorts 1 and 2 


Outcome 

P-Value for 

SEED Non-SEED Estimated Estimated 

Group Group Difference Difference 

Fourth year 





Standardized math score 

0.39 

0.19 

0.20 

0.257 

At or above proficiency level in math (%) 

66.7 

52.9 

13.8 

0.168 

Standardized reading score 

0.18 

0.24 

-0.06 

0.711 

At or above proficiency level in reading (%) 

61.1 

60.1 

1.0 

0.914 

Four-year high school graduation (%) 

69.3 

74.1 

-4.8 

0.592 

Sample size 

121 

93 




SOURCES: MDRC’s calculations using OSSE state test scores from the 2010-2011 and 2011- 
2012 school years. 

NOTES: Cohorts 1 and 2 consist of 214 students who were sixth-graders in the springs of 2007 
and 2008, respectively. 

Values for SEED group members are the simple means or percentages. Values for the 
difference between SEED group members and non-SEED group members are obtained from a 
regression of a given outcome on a series of indicator variables that identify each lottery, a 
student's baseline reading score, a student's baseline math score, indicators of family income, and 
an indicator variable that equals 1 for lottery winners and 0 for lottery losers. The coefficient on 
the latter indicator variable equals the difference in outcome between SEED group members and 
non-SEED group members. The value for non-SEED group members equals the corresponding 
value for SEED group members minus the estimated difference between SEED group members 
and non-SEED group members. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 


math exam than students in the non-SEED group, an effect equivalent in magnitude to 1 .4 times 
the typical annual gains for this age group. 13 According to OSSE’s proficiency benchmark, this 
increase in SEED group students’ math scores translated into a 13.8 percentage point increase in 
the proportion of students scoring at or above proficient. While the p-values of both these ef- 
fects on students’ standardized math scores are just above this report’s 0.10 benchmark, they 


13 Hill, Bloom, Black, and Lipsey (2008). 
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should not be discounted, as they are part of a pattern of positive math effects experienced by 
SEED group students. The effect of SEED on students’ reading scores in the fourth year is less 
clear, as the probability is high that the estimated effects are due to chance. 

The final row of Table 3.6 indicates that being offered the opportunity to attend SEED 
did not have an effect on students’ probability of four-year high school graduation. Specifically, 
69.3 percent of the students who were offered the opportunity to attend SEED as seventh- 
graders graduated from high school after four years (six years after the SEED lottery), while 
74.1 percent of the students in the non-SEED group did so. 14 The difference between these two 
groups, 5 percentage points, is not statistically significant. 

Effects on Student Behavior and Attitudes 

In addition to positively affecting students’ academic achievement, SEED aims to pro- 
mote positive behaviors such as rigorous study habits and self-control while discouraging “risky” 
behaviors such as alcohol use and unplanned pregnancy (see Box 3.1 for more detail). Table 3.7 
indicates that SEED may have had small effects on some measures of student behavior, but only 
four are statistically significant. While SEED group students did not report having developed 
more rigorous study habits or organizational skills than non-SEED group students, they did re- 
port spending four more hours a week doing homework than non-SEED group students. Table 
3.7 also indicates that at the time of survey administration SEED group students may have used 
less tobacco in the past 30 days. At the same time, Table 3.7 also presents students’ scores on the 
recent risky behavior scale, which asked about specific behaviors, such as skipping school with- 
out permission, getting into a fight or argument with parents, or hitting someone, in the past three 
months. Here there is a statistically significant positive effect, indicating more frequent risky be- 
havior among SEED group students. There also seems to be a negative effect of SEED on stu- 
dent reports of “grit.” This measure seeks to gauge students’ ability to stick with things over the 
long term by asking how often new ideas distract from previous ones, whether the student identi- 
fies as being a hard worker, and whether the student finishes what he or she begins and is dili- 
gent. This measure has been found to be a predictor of students’ future academic success. 15 


14 Note that the four-year high school graduation rate computed for the SEED group student sample is 
based on students who entered a lottery for SEED in sixth grade. This graduation rate is calculated differently 
from the four-year high school graduation rate computed by OSSE based on SEED’S entering ninth-grade co- 
hort; that rate was 89 percent for the cohort of 2009-2010 ninth-graders and was unknown for the 2010-2011 
ninth-graders, because the cohort size fell below OSSE’s sample size threshold of 25. 

15 Duckworth, Peterson, Matthews, and Kelly (2007). 
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SEED Evaluation Final Report 
Table 3.7 


Estimated Effects of Being Offered the Opportunity to Attend SEED on 
Behavior Outcomes, Cohorts 1 and 2 






P-Value for 


SEED 

SEED 

Estimated 

Estimated 

Outcome 

Group 

Group Difference 

Difference 

School behaviors (l-4) a 





Score on study skills scale 

2.98 

2.95 

0.02 

0.800 

Score on organizational skills scale 

3.03 

3.01 

0.01 

0.917 

Time use b 





Hours spent doing homework per week 

9.14 

5.12 

4.02 * 

0.051 

Hours spent reading for personal pleasure per week 

5.96 

2.97 

2.98 

0.280 

Hours spent watching television per week 

10.62 

8.58 

2.04 

0.495 

Hours spent playing video/computer games per week 

4.45 

4.68 

-0.23 

0.894 

Likelihood of attending college (1-4) L 

3.74 

3.66 

0.08 

0.471 

Risky behaviors' 1 





Score on recent frequency of risky behavior (1-5) 

1.72 

1.33 

0.39 ** 

0.030 

Practiced risky behavior (%) d 

15.1 

18.7 

-3.6 

0.515 

Had a baby/ fathered a baby (%) 

26.7 

30.9 

-4.2 

0.857 

Alcohol use in past 30 days (l-7) e 

1.96 

2.28 

-0.32 

0.291 

Tobacco use in past 30 days (l-7) e 

1.87 

2.55 

-0.67 * 

0.100 

Alcohol use in lifetime (l-7) f 

2.18 

2.59 

-0.40 

0.214 

Marijuana use in lifetime (l-7) f 

2.75 

3.25 

-0.50 

0.277 

Metacognitive skills (l-5) a 





Score on self-control scale 

3.55 

3.62 

-0.08 

0.553 

Score on grit scale 

3.50 

3.73 

-0.23 * 

0.080 

Sample size 

121 

93 



(continued) 
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Table 3.7 (continued) 


SOURCE: MDRC calculations based on data collected in student survey. 

NOTES: Cohorts 1 and 2 consist of 214 students who were sixth-graders in the springs of 2007 and 2008, 
respectively. 

Values for SEED group members are the simple means or percentages. Values for the difference between 
SEED group members and non-SEED group members are obtained from a regression of a given outcome on a 
series of indicator variables that identify each lottery, a student's baseline reading score, a student's baseline 
math score, indicators of family income, and an indicator variable that equals 1 for lottery winners and 0 for 
lottery losers. The coefficient on the latter indicator variable equals the difference in outcome between SEED 
group members and non-SEED group members. The value for non-SEED group members equals the 
corresponding value for SEED group members minus the estimated difference between SEED group members 
and non-SEED group members. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are indicated as: ** 
= 5 percent; * = 10 percent. 

a See Appendix F for definitions of the scales used in this report. 

This item is based on the following range: 0-168 hours per week. 

This item is based on the following scale: 1: Definitely not; 2: Not very likely; 3: Sort of likely; 4: Very 
likely. 

d Percentage represents those who answered “yes” to one of four questions on the overall engagement in 
risky behavior scale. See Appendix F for details. 

This item is based on the following scale: 1: 0 days; 2: 1 or 2 days; 3: 3 to 5 days; 4: 6 to 9 days; 5: 10 to 
19 days; 6: 20 to 29 days; 7: All 30 days. 

This item is based on the following scale: 1: 0 days; 2: 1 or 2 days; 3: 3 to 9 days; 4: 10 to 19 days; 5: 20 to 
39 days; 6: 40 to 99 days; 7: 100 or more days. 


In conclusion, winning the lottery for SEED had a positive long-term effect on students’ 
math achievement but no effect on students’ long-term reading achievement or probability of 
four-year high school graduation. In addition, students who won the SEED lottery reported 
spending more time doing homework during the week but also engaging in slightly more risky 
behaviors, and they scored lower than students who lost the SEED lottery on a measure of grit. 
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Chapter 4 

Conclusion 


The preceding chapters present a mixed picture of the quality of SEED DC’s implementation 
and its impacts on student outcomes. For example, the evaluation team found that SEED DC 
creates a highly supportive enviro nm ent for its students, with many opportunities for students to 
interact with caring adults. But classroom observations, while limited, found few examples of 
innovative, motivating instruction, particularly at the high school level. 

Similarly, while the evaluation found that SEED DC led to significant, notable increas- 
es in standardized test scores and proficiency levels — particularly in math — the available evi- 
dence suggests that, among the subset of students whom researchers could follow through four 
years of high school, SEED DC did not increase the proportion who graduated on time. (This 
study’s follow-up period is too short to measure SEED’S impact on postsecondary outcomes.) 
There is also no strong evidence from the student survey that SEED DC affected key nonaca- 
demic outcomes such as the prevalence of risky behaviors. 


Some Factors to Consider 

Several factors are critical to the interpretation of these findings: the program’s cost, the charac- 
teristics of students targeted for enrollment, the local educational context, the number of years 
spent in the school, and quality of the program’s implementation. 

Cost 

Owing to its boarding school model, SEED costs at least twice as much per student as a 
traditional nonresidential school. The earlier study of SEED DC by Curto and Fryer also found 
that SEED significantly increased students’ test scores but questioned whether those impacts 
were large enough to justify its very high cost. 1 This evaluation does not include a benefit-cost 
analysis, but it does measure SEED’S impacts on four-year high school graduation and nonaca- 
demic outcomes such as teenage pregnancy and involvement in the justice system. These addi- 
tional outcomes, which were not measured in the earlier study, have the potential to produce 
large societal benefits that could offset SEED’S cost. At this point, however, there is little evi- 
dence that SEED DC has affected either graduation or nonacademic outcomes, though it is im- 
portant to note that the sample size for measuring these outcomes (including only about 200 
students in cohorts 1 and 2) is quite small. In addition, it is possible that longer follow-up would 
reveal additional impacts. For example, while SEED students may not be more likely to gradu- 

'Curto and Fryer (2011). 
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ate high school, they may be better prepared for success in college. Nonetheless, based on the 
data available to date, the cost-effectiveness question raised by Curto and Fryer still seems high- 
ly relevant. 

Enrollment Targeting 

SEED’S founders argue that the intensive, holistic boarding school model is needed for 
some students who face very serious obstacles to school success in their homes and communi- 
ties. And, indeed, both the Maryland and Florida SEED schools are open only to students who 
meet certain specific criteria signifying severe disadvantage. In contrast, the DC school, the fo- 
cus of this evaluation, is open to any student who resides in the District of Columbia. In general, 
the baseline characteristics of students in the study sample resemble those of the DC public 
school population as a whole during the study period — most students are African-American 
and low-income, and just under half were proficient in reading and math. In addition, test score 
data for the fourth year of follow-up (generally tenth grade) show that reading proficiency rates 
for the non-SEED group were roughly 17 percentage points higher and mathematics proficiency 
rates roughly 12 percentage points higher than for DC students overall. In other words, it does 
not appear that SEED DC is serving a particularly disadvantaged segment of the DC school 
population. It is possible that SEED’S model would produce larger impacts for students facing 
more serious obstacles to success. 

Local Educational Context 

In any rigorous evaluation it is critical to examine the “counterfactual,” or what would 
have occurred in the absence of the program, as illustrated by the services received by the con- 
trol group. The District of Columbia has many innovative charter and magnet schools, and it 
appears that a large proportion of the students who lost the SEED lottery enrolled in these 
schools, particularly by the time they reached high school, when two-thirds of the non-SEED 
group were attending charter or magnet schools. It is possible that SEED would make a bigger 
difference in a context with fewer innovative alternatives. This point may also be related to the 
previous point: SEED may be serving a relatively motivated group of families who are likely to 
seek out other innovative options if they lose the lottery. Indeed, while one might assume that 
parents or guardians who are willing to send their child to a boarding school in sixth or seventh 
grade might be particularly concerned about the quality of their home life or neighborhood, it is 
equally likely that these parents are extremely motivated to ensure that their child has access to 
the best, most supportive school available to them. 
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Length of Stay at SEED 

SEED DC’s impacts on academic proficiency are substantial, particularly in the middle 
school years, and other literature suggests that middle school test scores are highly predictive of 
high school graduation. And yet there is no evidence to date that SEED has increased high 
school graduation rates. This fact, coupled with data showing that a sizable portion of students 
left SEED before twelfth grade, raises the question of whether SEED could have larger, more 
sustained impacts if more students remained in the school longer and received a larger “dose” of 
SEED — in other words, whether the school needs to focus mainly on promoting retention. 
This question is a difficult one to answer. We do not know exactly why lottery winners left 
SEED, or how they would have performed if they had stayed. For example, some of those who 
left may have decided that they wanted a more traditional high school environment and would 
have seen their performance decline if they had stayed at SEED. On the other hand, since SEED 
had a large effect on students in their middle school years, if students had stayed they might 
have experienced continued positive effects. 

Implementation Quality 

Impacts are driven not only by a program’s model but also by how well that model is 
implemented in reality. While the implementation study noted many positive qualities of SEED 
DC’s operation, it also raised questions about the quality of instruction in SEED DC’s class- 
rooms — particularly at the high school level — and noted that many students seemed to be 
struggling with the transition from eighth grade to ninth grade, even though they remained at the 
same school. The SEED Foundation has recently announced a renewed and intensified focus on 
the quality of instruction, leadership, and services at its schools. For example, the foundation 
reports that SEED DC has identified new curriculum resources to strengthen the middle school 
math program and is seeking to increase student engagement through interactive learning tech- 
nologies. Given these and other ongoing changes, it is possible that impacts on student out- 
comes — and, perhaps, retention rates — will be stronger in the future. The SEED Foundation 
reports that its new approach is being implemented successfully in its new Miami school. For 
more information about that school and its early experiences, see Appendix A. 


Summary 

SEED DC’s significant, positive effects on students’ academic achievement are notable. But 
given the lack of strong evidence of impacts on high school graduation or key nonacademic 
outcomes, the question about SEED’S cost-effectiveness, raised in earlier literature, remains 
relevant. 
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Similarly, the evaluation leaves open the difficult question of whether a boarding envi- 
ronment is appropriate or necessary for disadvantaged students to succeed. Some have criticized 
this approach for its implicit assumption that certain low-income parents are not able or willing 
to provide a sufficiently supportive environment for their children, and have suggested that re- 
moving children from their home may cause stress to families. Others note that low-income 
children should have access to the same kinds of intensive, round-the-clock supports that upper- 
income children can obtain at private boarding schools, and that in some cases having an ado- 
lescent child live away during the week may reduce stress in the home. 

With all that said, it is important to note that these results pertain only to a specific peri- 
od in the life of SEED DC, which operates in a particular context and, owing to local rules, is 
unable to give priority to the most disadvantaged students. Thus, the evaluation does not speak 
to the question of whether SEED’S unusual boarding school model can produce larger effects in 
different environments, with a more targeted group of students. 
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Appendix A 

School Program Improvements, As Described by SEED 




The SEED Foundation 


SEED School Growth 

In the past year, SEED expanded beyond Washington, D.C., and Maryland and opened its third school in 
Miami, Florida. Many lessons learned from SEED'S work with students in D.C. and Maryland were 
applied to the program at SEED Miami. 

On August 17, 2014, SEED Miami opened with its first class of incoming sixth graders. The demographic 
breakdown of the founding sixth grade class included African American, Hispanic, and Haitian students. 
The majority of students in this group were from Miami-Dade County, with a select few from Broward 
County. Of the 62 entering students in fall of 2014, 56 remained enrolled for the entire school year. 

SEED Miami recruits students specifically from South Florida's low-income underserved communities. 
Student eligibility criteria defined by Florida statute requires: residency of Florida and eligibility to 
attend school in participating school districts; eligibility for sixth grade during upcoming school year and 
born on or after February 1, 2003; family gross income at or below 200% federal poverty line; and 
eligibility for benefits or services funded by Temporary Assistance for Needy Families (TANF) or Title IV-E 
of the Social Security Act. 

In addition to the eligibility criteria, students must also meet at least one of the following at-risk factors: 
coverage under the terms of the state's Child Welfare Waiver Demonstration project; in foster care or 
declared an adjudicated dependent; current head of household is not the student's custodial parent; 
resident in a household receiving a housing voucher or eligible for public housing; or an immediate 
family member has been incarcerated. Furthermore, a third of the open slots at SEED Miami are 
reserved for children who have received or are receiving services from the child welfare system. 

Organizational Design 

Customizing its support to better serve SEED students, SEED Miami has a unique organizational 
structure. Unlike SEED DC and SEED MD, SEED Miami has a president who focuses on all operations and 
external relations functions such as board management, facilities, legislative advocacy, communications, 
and fundraising. This structure expands the capacity of the head of school to stay focused on the 
programmatic needs and outcomes of SEED'S 24-hour learning model. 

In addition to this strategic shift in organizational structure, the general process of hiring programmatic 
leaders for new schools has also been systematized. First, the head of school is hired approximately 14 
months prior to the opening of the school. Then, academic and student life directors are hired nine 
months before the schools opens. While a thorough search is conducted for leadership roles, SEED 
recognizes the importance of maintaining institutional knowledge and fidelity to the SEED way. Thus, 

SEED has nurtured internal talent to develop strong candidates for future leadership roles; the heads of 
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school at SEED Miami and SEED MD were long-time leaders in SEED schools before transitioning to the 
head of school role. In addition to establishing a thorough process for identifying strong leadership, 

SEED also recognizes the importance of incorporating a year-long planning process prior to opening a 
new SEED school. This process enables school leaders to thoughtfully prepare to recruit students from 
underrepresented communities, implement SEED'S 24-hour learning program, and establish a 
performance plan to meet SEED'S network-wide goals and its school-specific goals. 

SEED Miami has also been deliberate in its approach to hiring to ensure that strong candidates are 
clearly aligned with the SEED model; in fact, mission-fit is a key criterion that is assessed at each stage in 
the application process. For the 2015-2016 SY, 95% of SEED Miami faculty who were asked to return 
have pledged to do so next year. 

Program Improvements 

Based on academic lessons learned from SEED'S work with students in D.C. and Maryland, SEED Miami is 
directly addressing the need for writing support by deliberately separating the reading and writing 
curriculum. At SEED Miami, students are required to take three writing interim assessments prior to the 
administration of the Florida Standards Assessment (FSA). 1 The writing interim assessments cover the 
following three categories: (1) purpose, focus and organization, (2) evidence and elaboration, and (3) 
writing conventions. Each writing assessment requires students to respond to prompts using textual 
evidence from 2-3 provided supporting documents. These interim assessments are modeled after the 
FSA to provide staff with more information on which areas challenge students the most. These interim 
results inform instructional practices and form the basis of customized lesson plans. The trends in the 
fall and winter assessment scores this year indicate that students' performance increased overall in all 
three writing categories. Further evidence of student improvement due to writing curriculum changes 
are reflected in the SEED Miami Spring 2015 ACT Aspire Summative writing assessment results. ACT 
Aspire Summative assessments are annual exams that measure a student's proficiency in subject 
content to determine if a student is on track to be college- and career- ready. Compared to the sixth 
grade national average of 24% proficiency in writing, 40% of sixth grade SEED Miami students are 
proficient in writing according to ACT Aspire. 

In addition to the curriculum improvements, other program updates are being implemented at SEED 
Miami. Notably, a six week onboarding process for all new programmatic employees offers orientation 
to the SEED mission and belief statements, Common Core and State Standards training, data analysis 
and action planning along with developmental assets overview. Also, regular home visits are conducted 
to promote the partnership between SEED and students' families. Moreover, data is used intensively on 
a new platform to share students' progress and, when necessary, to develop interventions accessible to 
families, students and SEED staff. 


^he Florida Standards Assessments is Florida's K-12 assessment system which measures student achievement of the state's 
academic standards. These standards were developed to ensure students are prepared for success in college, career and life. 
Florida Department of Education, "Florida Standards Assessment" (2014), http://www.fldoe.org/accountability/assessments/k- 
12-student-assessment/fsa.stml. 
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First-Year Outcomes 


Within the reading curriculum, SEED Miami students are assessed on multiple factors, including reading 
level growth and English Language Learners' (ELL) proficiency in English. Students' reading level growth 
is assessed at the school level using the Fountas and Pinnell (F&P) reading system. SEED Miami assesses 
student reading levels at least four times during the school year; this is in addition to the New Student 
Orientation baseline assessment administered in July 2014. Despite reading three years below grade 
level, on average, at the time of enrollment, students at SEED Miami improved by one reading level in 
their first three months at SEED. Furthermore, all students grew two or more reading levels during their 
first year as depicted in Figure l. 2 3 


The SEED School of Miami 


F&P Reading Level Growth During 2014-2015 SY 



Growth in Reading Levels 


Figure 1. According to the Fountas and Pinnell model, the pace and amount of growth in a single year 
depends on the starting reading level for each student and the teaching the student receives. Each 
successive level of the gradient makes greater and more varied demands on the reading process. 
Students typically advance through at least a few levels each year, but as the levels increase, so do texts' 
difficulty; advancement through the higher levels sometimes takes longer than through the lower levels. 3 


In addition to F&P assessments, SEED Miami students also take the Comprehensive English Language 
Learning Assessment (CELLA). CELLA is a tool that the state of Florida utilizes to measure the progress of 
English Language Learners' proficiency in English. This district-mandated, standardized assessment is 
administered every spring to ELL students to measure performance in reading, listening, and writing. 
Seven SEED Miami sixth graders took the CELLA exam in spring 2015. Figure 2 (below) compares 


2 As cited in the SEED Miami Board of Trustees Report for 2014-2015 SY, Quarter 4. This chart totals 63 students which includes 
all children who were assessed more than one time (to include 1 withdrawn student). 

3 Source: http://www.heinemann.com/fountasandpinnell/pdfs/whitepapertextgrad.pdf. 
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students' fifth grade scores prior to enrolling in SEED from spring 2014 to their sixth grade scores from 
spring 2015. 


The SEED School of Miami 
2014 vs. 2015 CELLA Score Comparison 



Figure 2. Overall, these seven SEED Miami sixth graders experienced overall gains in reading and 
individual growth in writing proficiency. Moreover, the seven sixth graders that took CELLA in spring 
2015 scored, on average, within the high intermediate range for reading and writing, meaning students 
are reading in English at grade level with minimal support. 
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Appendix B 

Implementation Research Data Collection and Analysis 




Data Sources 

To understand how SEED DC is structured and how it operated in practice during the three-year 
implementation study period, MDRC researchers conducted two-day site visits to SEED DC in 
spring and fall 2012 and fall 2013. During these site visits the research team requested and was 
granted interviews with staff members who had leadership and supervisory responsibilities for 
the various departments and programming, including the head of school, principal, director of 
Student Life, director of Student Support Services, director of College Counseling, middle and 
high school directors, and evening academic intervention coordinator, among others. MDRC 
also requested and was granted access to a cross section of the faculty, representing different 
grade levels and subject areas, recruited by SEED for the interviews. Finally, MDRC requested 
that SEED select a cross section of middle school, ninth-grade, and high school students to par- 
ticipate in grade-level focus groups. In some cases interviews were conducted by phone. These 
data collection activities were supplemented by interviews and focus groups with SEED Foun- 
dation staff members. Before these visits, researchers reviewed a variety of documents from the 
SEED Foundation and SEED DC, but the bulk of data collection occurred on site. 1 

In total, the team conducted interviews with 10 members of the SEED Foundation staff, 
22 members of the school’s administration and staff, and 13 members of the school’s faculty. 
(Over the course of the study, some individuals were interviewed more than once.) The team 
also “shadowed” three middle school students, one ninth-grade student, and three high school 
students, which involved accompanying the students to class and other activities, touring their 
donn rooms, and eating lunch with the students and their friends. 

To break the process down by site visit, in spring 2012 the team conducted six inter- 
views with SEED administrators and staff members and one focus group with two teachers. The 
team conducted observations of eight academic classes as well as other activities, such as the 
admissions lottery, student Presentations of Learning, Targeted Instruction, a junior-to-senior 
transition ceremony, a scholarship ceremony, and physical education. Four student shadows 
were also conducted. 

During the fall 2012 visit, the team interviewed six SEED administrators and staff 
members and conducted one focus group with five Student Life staff members and one with 
four middle school students. The students had enrolled in SEED in sixth grade and had been 
participating in the program for one to two years. The visit also included 10 classroom observa- 
tions; an observation of Targeted Instruction; observation of three Student Life activities; one 
observation of outdoor, experiential education; and three student shadows. 


'Following each site visit, MDRC provided the SEED Foundation with feedback memos that described 
the research team’s early impressions. 
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In fall 2013, the team conducted 10 phone interviews with SEED administrators and 
staff members. The fall 2013 school site visit included several focus groups: one with three 
middle school teachers, one with three ninth-grade teachers, one with three high school teach- 
ers, one with four instructional coaches, and one with five Student Life staff. Three student fo- 
cus groups (one with four middle school students, one with six ninth-grade students, and one 
with six high school students) were also conducted. Most student participants had enrolled in 
the school in sixth grade, so middle school focus group participants had been at SEED for one 
to two years, ninth-grade students had been at SEED for three years, and high school students 
had been at SEED for four to six years. The site visit also included two observations of Student 
Life activities. 


Data Collection and Analysis 

As detailed above, MDRC researchers used semistructured interview and focus group protocols 
to collect qualitative data from SEED DC administrators, staff members, and faculty members 
and from SEED Foundation staff members. The purpose was to understand the school context, 
school structure, and adult perspectives on the key program components (academics, student 
life, support services, and college counseling) and to leam how these components interact. 
(Sample interview protocols appear at the end of this appendix.) These data also captured im- 
plementation changes from year to year. 

Similarly, student focus groups were conducted to gain an understanding of student 
perceptions about SEED DC and to leam more about students’ experiences at the school. The 
research team used focus group protocols to ask students about their decision to attend the 
school and the characteristics they found attractive about the school, their views on single- 
gender classes and the rigor of the curriculum, their experiences transitioning between middle 
school and high school, the role of Student Life and Student Support Services, their exposure to 
external opportunities, how they believed SEED prepared students for college, and the weekly 
transition to and from home on the weekends. (A sample high school focus group protocol fol- 
lows the interview protocols at the end of this appendix.) As the students were chosen by 
SEED, the research team cannot know whether they were representative of the population as a 
whole. The research team did not compensate students, staff members, or faculty members for 
participating in the focus groups and interviews. 

Informal classroom observations were conducted to gain a general understanding of 
how classes are structured, how teachers and students interact, and the types of instructional 
practices being used. In most cases, classes were observed for the full class period, although 
some classes were observed for less than the full period. In one instance a pair of researchers 
observed the same classroom, but in most instances each classroom was observed by one re- 
searcher. A formal observation rubric was not used. Instead, the researchers recorded their ob- 
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servations using handwritten notes that captured information about the structure of the class- 
room, the content covered, and the nature of teacher and student engagement, including class- 
room management practices tied to the use of the “SEED note” (in middle school classes) and 
the SEED core values, as described in Chapter 2. Observers also took notes on the kinds of in- 
structional practices being used in the classroom. Following the observations, the team dis- 
cussed what had been observed, identified themes across the set of observations, and identified 
clarifying questions for the SEED Foundation staff and SEED DC administrators and faculty. 
While the observations may be considered subjective and are based on a small number of mid- 
dle school and high school classrooms, they are included to provide a general set of impressions 
about the SEED DC classroom experience on the days when classes were observed. Observa- 
tions of academic classes were conducted in 10 middle school classrooms and 12 high school 
classrooms. 

In addition to conducting interviews, focus groups, and observations, researchers re- 
viewed school and foundation documents. In the early stages of data collection, these data 
served primarily as background infonnation on the SEED model and assisted the team in de- 
signing implementation data collection tools for site visits. The documents included an organi- 
zational chart, descriptions of SEED DC’s organizational departments, descriptions of staff po- 
sitions, SEED’S belief statements, two research studies commissioned by the SEED Foundation 
on college success and boarding school models, and an overview of the SEED blueprint docu- 
ment process. Further along in data collection, the team requested concrete examples of student 
schedules, in particular to understand the Student Life component. Staff members at SEED DC 
provided these documents following interviews. Similarly, SEED Foundation staff members 
provided documentation about SEED’S college matching approach after a phone interview on 
the topic. 

Team members took notes during all data collection activities, and focus groups and in- 
terviews that were conducted in fall 2013 were audio-recorded and transcribed. The team did 
not use a qualitative analysis software program to analyze these data; instead, they were sorted, 
coded, and analyzed manually. The process for coding the interviews and focus groups involved 
multiple steps: First, all documents were reviewed: interview and focus group transcripts, which 
ranged from 15 to 40 pages, and interview and observation notes, which were less than 10 pages 
per session. Second, notes were taken on each document to highlight ideas and topics covered. 
This post-data collection documentation served as a general outline of relevant themes men- 
tioned in each document and a useful reference guide to identify common themes across multi- 
ple data sources that could be coded together. The guide also helped researchers easily navigate 
the various data sources. Third, general codes were developed and assigned to specific text in 
each document. General codes reflected general themes outlined in the protocols (for example, 
academic support) and relevant themes that emerged from the data. Fourth, the general codes 
were reviewed and refined to represent subtopics within the larger theme (for example, academ- 
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ic support for struggling students). On occasion, these subtopics were further refined (for exam- 
ple, tutoring). Themes that emerged across more than one data source were combined, allowing 
researchers to analyze the data from the perspective of multiple informants (such as faculty and 
students) and to conduct an analysis using multiple data sources (such as interviews and obser- 
vations). This multistage process of sorting and coding the data informed the analysis presented 
in this report. 


Examples of Interview and Focus Group Protocols 

The following pages present interview protocols for the SEED DC head of school, principal, 
director of Student Life, high school program director, and director of College Counseling, and 
the high school student focus group protocol. 
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SEED Evaluation 

Interview Protocol for SEED DC Head of School 


Introduction 

• Thank respondent for his/her time. 

• Interviewer(s) introduces self and study. 

• Describe purpose of interview. (Mention follow-up to May 2012 interview.) 

• Confirm time allotted for interview. 

• Ask for permission to record the interview. 

Roles and duties 

1) We want to make sure we understand all of your roles and responsibilities at the school. 

a. Please describe your roles and responsibilities for us. 

2) In terms of reporting structures: 

a. Who is your immediate supervisor? 

b. Which staff members report to you? 

3) Please describe the recent and impending changes to the DC School’s leadership. 

a. What precipitated these changes? 

b. Is there a transition plan in place to replace the principal? 

c. What is the role of the Managing Director? 

d. Are these changes reflected in an existing organizational chart that can be 
shared with us? 

Programmatic Leadership Team (PLT) structure 

4) We want to make sure that we understand the structure of SEED’S programmatic lead- 
ership. 

a. Please describe the composition of the Programmatic Leadership Team (PLT). 

i. How often does the PLT meet? 

ii. Who is responsible for leading the meeting? 

iii. What topics or issues are discussed in the meeting? 

Programmatic content 

5) Does SEED have a specific philosophy about programming? 

6) What are the goals of the school’s student programming? 

a. What are the specific goals of the model of care? 

7) In your opinion, is the SEED programmatic experience different than that of other 
schools in the area? 

a. How does it differ? 
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8) How does the SEED programmatic approach support a “college prep” curriculum? 
a. Probe: 

i. In terms of programmatic activities. 

ii. In terms of promoting a college -going culture. 


Student Life 

9) Please describe the role of the following in the SEED model: 

a. Student Life programming 

b. Gender-specific activities 

10) Can you please explain the founders’ desire to make SEED a boarding/residential 
school? 

a. Do you believe it is effective? 

i. If yes, why 

ii. If no, why not 

11) Have you been satisfied with the Student Life restructure implemented last year? 

a. What has worked well? 

b. Do area for improvement still exist? 

Student Support Services 

12) How do you think the Student Support Services component of affects students’ experi- 
ence at SEED? 

Goals for SEED students’ academic and postsecondary preparation and transition 

13) Does SEED have a specific philosophy about postsecondary readiness, access and suc- 
cess? 

14) What are the school’s postsecondary goals for students? 

a. Is there an expectation that all students will attend college? Why or why not? 

b. Are there specific types of colleges that SEED targets for its students? Why? 

15) Are Student Life staff expected or instructed to talk about college and college-going in 
a certain way? 

16) How many students take the PSAT and/or the SAT/ACT in preparation for applying to 
college? 

17) Please describe the college matching process at SEED. 

a. Who developed the college-matching process at SEED? 

18) In general, do you feel that SEED is successful in its college -preparatory mission? 

a. Are there areas in which you’d like the school to improve? 
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1 9) How do SEED students fare in college? 

a. What do the data suggest about postsecondary retention and graduation? 

b. Are you satisfied with the data? 

i. If no, what steps is the school taking to improve upon the data? 

20) What specific supports are in place for students who do not plan to attend college? 

Students 

Let’s discuss the average SEED student. 

2 1 ) How would you describe the SEED student population? 

a. Probe: 

i. In terms of demographics (race, gender, SES) 

ii. Do they represent a specific geographic location? 

22) Does the school target students during recruitment? 

a. Are students recruited from a specific set of elementary schools? 

b. Does SEED make special efforts to enroll “at-risk” students? 

i. If yes, how do you define “at-risk”? 

23) How would you describe the rate of enrollment from winning the lottery? 

a. When someone applies and is offered a spot, do they generally end up attend- 
ing? 

b. How do parents respond to the boarding component? 

i. Do they view it as an asset? A deterrent? 

24) In your opinion, what makes SEED attractive to parents? 

25) Once enrolled in SEED, are there a set of challenges that SEED students often or typi- 
cally face? 

a. Probe: 

i. In terms of academics 

ii. Boarding feature 

iii. Lack of exposure to more diverse student populations 

1 . Can you tell us more about the school’s efforts to promote ex- 
posure to other students and experiences outside of SEED? 

26) What do you think is(are) the biggest challenge(s) facing SEED students? 

a. Probe: 

i. How does the school attempt to address these issues? 
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27) What proportion of an entering cohort of 6th graders remains through graduation? 

a. Probe: 

i. For what reasons do students typically leave? 

ii. What proportion of students is asked to leave the school? 

Family engagement 

28) What are parents’/guardians’ and families’ expectations for their child at SEED? 

a. Why do you think they choose SEED? 

29) Flow involved are most parents throughout the year? 

30) Flow does SEED communicate with parents about students’ progress? 

31) Does SEED monitor students and parents’ usage of Power School to keep track of stu- 
dent progress? 

a. Do you think parents are using this opportunity? 

Foundation relationship 

Let’s discuss your working relationship with the Foundation. 

32) Flow often are you in communication with Foundation staff? 

a. Who is your main contact(s)? 

b. In what specific ways does the Foundation support your work as Plead of 
School? 

c. Are you working with Mary Lease at the Foundation in the development of the 
SEED Blueprint? 

i. Please tell us about this process. 

Conclusion 

33) What are your biggest challenges as Elead of School? 

a. Probe 

i. Related to students’ academic success. 

ii. Related to leading a boarding school. 

iii. Related to leading a charter school. 

34) What are the key challenges you expect the school to face in the future? 

35) What have been your biggest successes/accomplishments to date as Plead of School? 

36) Is there anything we haven’t talked about that would be important for us to know about 
SEED and its progress to date? 

Thank you for your time. 
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SEED Evaluation 

Interview Protocol for SEED DC Principal 


Introduction 

• Thank respondent for his/her time. 

• Interviewer(s) introduces self and study. 

• Describe purpose of interview. (Mention follow-up to May 2012 interview.) 

• Confirm time allotted for interview. 

• Ask for permission to record the interview. 

Roles and duties 

1) We want to make sure we understand all of your roles and responsibilities at the school. 

a. Please describe your roles and responsibilities for us. 

2) In terms of reporting structures: 

a. Who is your immediate supervisor? 

b. Which staff members report to you? 

Academic staffing structure 

3) We want to make sure that we understand the structure of SEED’S academic leadership. 

a. Please describe the composition of the Academic Leadership Team ( ALT). 

b. Do the instructional coaches participate? 

c. Do the program directors participate? 

4) Please explain the roles of the instructional coaches. 

a. How many are there? 

b. How are these individuals chosen? 

c. In what specific ways do you support their work? 

5) Please explain the roles of the two program directors (Jessica and Matthew). 

a. How are these individuals chosen? 

b. In what specific ways do you support their work? 

6) In what specific ways do the program directors interact with the instructional coaches? 
And with the teachers? 

a. Probe: Are there formal structures in place to facilitate collaboration? 
i. If yes, describe. 

7) We’d like to understand the nature of the ALT meetings. 

a. How often does the ALT meet? 

b. Who is responsible for leading the meeting? 
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c. What topics or issues are discussed in the meeting? 

d. Since ALT participants have different expertise and grade level focus: 

i. Does this present any challenges? Please describe. 

ii. Does convening a mix of staff promote success? How? 

8) Does SEED have a special education coordinator? 

a. Does this person participate in the ALT? 

b. What are her/his roles and responsibilities? 

c. How does she/he coordinate services with the teachers? 

Academic content 

9) Does SEED have a specific philosophy about academics or a specific academic focus? 

1 0) What are the goals of the academic program at SEED? 

a. Probe: What are the goals for students at each grade? 

b. What are the goals for students at the school level (DC-CAS scores, etc.)? 

11) What academic courses are students required to take in order to graduate? 

12) Does SEED offer a differentiated curriculum? 

a. AP or Honors classes? 

b. Developmental education (remedial)? 

c. Basic skills? 

13) Are students able to participate in dual-enrollment programs or take college courses 
while still in high school? 

a. If yes, at what postsecondary institution? 

b. If yes, what courses do they take/programs do they participate in? 

14) We’d like to learn about the content that students are covering in each grade level: 



6th grade 

7th grade 

8th grade 

9th grade 

1 0th grade 

1 1th grade 

12th grade 

Math 








Language 

Arts/English 








Social Studies 








Science 
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15) How do teachers and/or instructional coaches develop their curriculums? 

1 6) Are you involved in curriculum development? 

a. In what specific ways? 

17) Has the Common Core been integrated into the SEED curriculum? 

18) Are you working with Mary Lease at the Foundation in the development of the SEED 
Blueprint? 

a. Please tell us about this process. 

19) Does the Foundation direct the curriculum or academic goals of the school? 

a. In what specific ways 

20) In your opinion, is the SEED academic experience different than that of other schools in 
the area? 

a. How does it differ? 

2 1 ) How does the SEED approach support a “college prep” curriculum? 

a. Probe: 

i. In terms of specific course work. 

ii. In terms of promoting a college -going culture. 

Academic supports for students 

22) What academic supports are available to all SEED students? 

23) What supports are available to students who may be struggling academically? 

24) Please describe the tutoring support provided. 

a. How does this work? 

25) Please describe the TAP system. 

a. How does this work? 

Student Life and academics 

26) How do you think the Student Life component of SEED affects students’ academic ex- 
perience? 

a. Probe: 

i. Directly (ex. tutoring time) 

ii. Indirectly (ex. developing social skills) 
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27) Please describe the role of the following in the SEED model: 

a. Structured homework time in the evenings 

b. Structured opportunities for study/reading time 

c. Gender-specific activities 

28) How much does Student Life staff know about the academic life of students? 

a. Probe: 

i. Their academic subjects 

ii. Their assignments 

iii. How do they learn this information? 

29) Do you think the boarding school model is effective? 

a. Why? If yes, in what ways? 

b. If no, why not? 

Student support services and academics 

30) How do you think the Student Support Services component of SEED affects students’ 
academic experience? 

Goals for SEED students’ academic and postsecondary preparation and transition 

31) Does SEED have a specific philosophy about postsecondary readiness, access and suc- 
cess? 

32) What are the school’s postsecondary goals for students? 

a. Is there an expectation that all students will attend college? Why or why not? 

33) Are staff (teachers) expected or instructed to talk about college and college -going in a 
certain way? 

34) How many students take the PSAT and/or the SAT/ACT in preparation for applying to 
college? 

35) Are there specific types of colleges that SEED targets for its students? Why? 

36) Please describe the college matching process at SEED. 

a. Who developed the college-matching process at SEED? 

37) In general, do you feel that SEED is successful in its college-preparatory mission? 

a. Are there areas in which you’d like the school to improve? 

38) How do SEED students’ fare in college? 

a. What do the data suggest about postsecondary retention and graduation? 
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b. Are you satisfied with the data? 

i. If no, what steps is the school taking to improve upon the data? 

39) What specific supports are in place for students who do not plan to attend college? 

Students 

Let’s discuss the average SEED student. 

40) How would you describe the SEED student population? 

a. Probe: 

i. In terms of demographics (race, gender, SES) 

ii. Do they represent a specific geographic location? 

41) Are students drawn from a specific set of elementary schools? 

42) How would you describe the rate of enrollment from winning the lottery? 

a. When someone applies and is offered a spot, do they generally end up attend- 
ing? 

b. How do parents respond to the boarding component? 

i. Do they view it as an asset? A deterrent? 

43) In your opinion, what makes SEED attractive to parents? 

44) Once enrolled in SEED, are there a set of challenges that SEED students often or typi- 
cally face? 

a. Probe: 

i. In terms of academics 

ii. Boarding feature 

iii. Lack of exposure to more diverse student populations 

45) During our last interview you mentioned a desire to increase SEED students’ exposure 
to people from different backgrounds. Can you tell us more about the school’s efforts to 
promote exposure to other students and experiences outside of SEED? 

46) What do you think is the biggest challenge facing SEED students? 

47) How does the school attempt to address these issues? 

Family engagement 

48) What are parents’/guardians’ and families’ expectations for their child at SEED? 

a. Why do you think they choose SEED? 
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49) How involved are most parents throughout the year? 

50) How does SEED communicate with parents about students’ progress? 

51) Does SEED monitor students and parents’ usage of Power School to keep track of stu- 
dent progress? 

a. Do you think parents are using this opportunity? 

Foundation relationship 

Let’s discuss your working relationship with the Foundation. 

52) How often are you in communication with Foundation staff? 

a. Who is your main contact(s)? 

b. In what specific ways does the Foundation support your work as principal? 


Conclusion 

53) What are your biggest challenges as principal? 

a. Probe 

i. Related to students’ academic success. 

ii. Related to leading a boarding school. 

iii. Related to leading a charter school. 

54) What have been your biggest successes to date as principal? 

55) Is there anything we haven’t talked about that would be important for us to know about 
SEED and its progress to date? 

Tha nk you for your time. 
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SEED Evaluation 

Interview Protocol for SEED DC Director of Student Life 


Introduction 

• Tha nk respondent for his time. 

• Interviewer(s) introduces self and study. 

• Describe purpose of interview. 

• Confirm time allotted for interview. 

• Ask for permission to record the interview. 

Roles and duties 

1) We want to make sure we understand all of your roles and responsibilities at the school. 

a. Please describe your roles and responsibilities for us. 

2) In terms of reporting structures: 

a. Who is your immediate supervisor? 

b. Which staff members report to you? 

i. Probe: Anyone in addition to the LSCs and RAs? 

1 . Tutors? 

2. Evening Academic Intervention Coordinator? 


Student Life (general) 

3) What are the main goals of the Student Life department? 

a. Probe: 

i. Are these different for the various age groups? 

ii. Specify. 

4) What specific skills do you hope students will acquire from the Student Life compo- 
nent? 

a. Probe: 

i. Are these different for the various age groups? 

ii. Specify. 

5) In your opinion, why is the Student Life component of SEED necessary for students? 

6) In your opinion, what do SEED students gain during Student Life programming that a 
student at a traditional public school would not? 

7) Does SEED have a specific philosophy about youth development that informs the way 
Student Life operates? 

a. What is this philosophy? 
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8) How does Student Life approach facilitating relationships between students and adults? 

a. Probe: 

i. Do students have formal mentors? 

ii. Informal mentors? 

9) In your opinion, how does the Student Life component prepare students for college? 

a. How does Student Life programming promote a college -going culture? 


Staffing 

10) How many LSCs are there per cohort? 

11) What are the main responsibilities of the LSCs? 

a. Since several LSCs are responsible for each cohort, how do they divide respon- 
sibilities? 

12) How many RAs are at the school? 

13) For how many students are the RAS responsible? 

14) What are the main responsibilities of the RAs? 

15) How do you coordinate with the Evening Academic Intervention Coordinator? 

1 6) Does the entire Student Life department meet on a regular basis? 

a. How often? 

Content of Student Life 

17) Please describe all of the activities, lessons, services that comprise the Student Life 
component of SEED. We would also like to know when these activities take place, 
how often students participate, how many students participate (is it required or option- 
al), and what the main goals of the activities are. 
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When does this 
happen during 
the day? 

How often do 
students partici- 
pate? 

How many stu- 
dents participate 
(is it required or 
optional)? 

What are the 
main goals of 
the activity? 

HALLS lessons 





Drop Everything 
and Learn 





Drop Everything 
and Read 





Sparks 





Tutoring 





Boys and Girls 
Clubs 





Study Zone 
















18) How are HALLS lessons developed? 

a. What staff is involved? 

b. Are the HALLS lessons developed at the beginning of the year or throughout 
the years? 

1 9) Are there specific content areas that are covered every year for each age cohort? 

a. If so, what are they? 

b. How was it decided that these were the necessary topics? 

20) Please tell us more about HALLS lessons. 

a. What do HALLS lessons look like? 

i. How long do they last? 

ii. What types of activities are usually included? 

1. Probe: 

a. Discussion? 

b. Projects? 

c. Lecture? 

iii. Who leads them? 
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21) What happens during Drop Everything and Learn time? 

22) What happens during Drop Everything and Read time? 

23) Please tell us more about Sparks. 

a. What are the Sparks that are available to students? 

b. Are they specific to age cohorts? 

c. How does a student learn about what Sparks are available to them? 

d. Who leads the Sparks? 

24) Please tell us more about tutoring at SEED. 

a. How is it decided that a student should attend tutoring? 

b. Who are the tutors? 

25) Please tell us more about the Boys and Girls Clubs. 

a. What happens during the time that they meet? 

b. Who leads these clubs? 

26) What happens during Study Zone? 

27) Does the Student Life component involve activities or lessons about college/college- 
going? 

a. If so, where does this happen? 

b. What topics are covered? 

c. How do you coordinate with the Director of College Counseling? 

d. Are Student Life staff members instructed to talk about college in any certain 
way? 

28) Does SEED have a student government? 

29) Are there opportunities for SEED students to participate in community service projects? 

a. If so, what types? 

30) Does SEED have a band or orchestra? 

31) Are there any other clubs or activities available to students? 
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Student Life and academics 

32) How do you thi nk the Student Life component of SEED affects students’ academic ex- 
perience? 

a. Probe: 

i. Directly (ex. tutoring time) 

ii. Indirectly (ex. developing social skills) 

33) How much does Student Life staff know about the academic life of students? 

a. Probe: 

i. Their academic subjects 

ii. Their assignments 

iii. How do they learn this information? 

34) Do you think the boarding school model is effective? 

a. Why? If yes, in what ways? 

b. If no, why not? 

Non-academic outcomes 

35) What non-academic outcomes does the Student Life component try to affect? 

a. Probe? 

i. Self-esteem 

ii. Grit 

iii. Self-control 

b. How does it try to affect them? 

c. Why do SEED students need activities that focus on these areas? 

36) How do you measure these non-academic outcomes? 

Conclusion 

37) What are your biggest challenges as Director of Student Life? 

a. Probe 

i. Related to students’ academic success. 

ii. Related to leading a boarding school. 

38) What have been your biggest successes to date as Director of Student Life? 
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SEED Evaluation 

Interview Protocol for SEED DC High School Program Director 


Introduction 

• Thank respondent for her time. 

• Interviewer(s) introduces self and study. 

• Describe purpose of interview. 

• Confirm time allotted for interview. 

• Ask for permission to record the interview. 

Roles and duties 

1) We want to make sure we understand all of your roles and responsibilities at the school. 

a. Please describe your roles and responsibilities for us. 

2) How long have you been working at SEED? 

3) In terms of reporting structures: 

a. Who is your immediate supervisor? 

b. Which staff members report to you? 

Academic Leadership 

4) Please describe your role on the Academic Leadership Team (ALT). 

a. Probe: 

i. Are there benefits to this structure? 

ii. Any “cons” to this structure. 

5) How does the principal support you in your work? 

6) How do you work with the Instructional Coaches? 

7) In what ways do you work with the Middle School Program Director? 

a. Probe: 

i. Do you meet regularly with him? 

ii. Do you coordinate tasks or activities? 

1. If yes, specify. 

Academic content and structure 

8) Does SEED have a specific philosophy about academics or a specific academic focus? 

a. Probe: 

i. Specifically for high school? 

ii. How were you introduced to this philosophy? 
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9) What are the goals of the academic program at SEED? 

a. Specifically for the high school? 

b. Probe: 

i. What are the goals for students at the student level (grades)? 

ii. What are the goals for students at the school level (DC-CAS scores, 
etc.)? 

10) We understand that middle school classes are separated by gender. 

a. Probe: 

i. In 9th grade, how do students cope with being in coed classes? 

11) How do teachers and/or instructional coaches develop their curriculums? 

12) Are you involved in curriculum development? 

13) Has the Common Core been integrated into the SEED curriculum? 

14) Are there specific curriculums that your teachers are using? In. . . 

a. ELA? 

b. Math? 

c. Science? 

d. Social studies? 

15) What are the main skills that high schoolers should have before transitioning to high 
school ? In... 

a. ELA? 

b. Math? 

c. Science? 

d. Social studies? 

1 6) What are the main skills that high schoolers should have before transitioning to college ? 
In... 

a. ELA? 

b. Math? 

c. Science? 

d. Social studies? 

17) How is college discussed in the high school grades? 

a. Probe: 

i. Are teachers instructed to talk about it in a certain way? 

ii. Is it a formal part of students’ high school experience? 

1. If so, how? 
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18) In your opinion, are students adequately prepared to attend a rigorous college at the end 
of their senior year? 

a. Probe: 

i. Academically prepared? 

ii. Socially prepared? 

iii. Emotionally prepared? 

Instruction and observation 

1 9) Are you responsible for observing teachers? 

a. Probe: 

i. If so, how many times per year? 

ii. How do you assess their instruction? 

iii. What are you looking for during your observations? 

iv. How do you provide feedback to the teachers? 

20) Are there specific instructional approaches that your teachers are using? 

a. ELA? 

b. Math? 

c. Science? 

d. Social studies? 

21) What is your involvement in the development and implementation of interim assess- 
ments? 

a. Probe: 

i. Do you participate in Data Days? 

22) How else do you support teachers? 

23) Are you responsible for supervising teachers’ implementation of the Model of Care? 

a. If so, how? 

SEED Students 

24) We learned that many middle school students arrive at SEED below grade level. 

a. Are they performing on grade level by the time they reach 9th grade? 

b. Do many students transfer into SEED at 9th grade? 

25) What are SEED high school students’ main academic challenges? 

26) How does the staff address these challenges? 
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27) Do you work closely with the Special Education Coordinator? 

a. If so, how? 


Teachers 

28) How do you ensure adequate planning time for teachers? 

29) What are the main challenges faced by your teachers? 

a. How and when do you learn about these challenges? 

b. How are these challenges addressed? 

30) Are you involved in determining or developing professional development opportunities 
for teachers? 

a. If so, what do you focus on? 

b. From what types of training do you think they benefit most? 

c. In what areas might more training be helpful? 

Coordination with other SEED components 

31) How do you interact with the other departments at SEED? 

a. Student Life 

b. Student Support Services 

c. College Counseling 

32) Are you involved in students’ evening academic programming? 

a. If yes, how? 

33) In your opinion, how does the boarding environment affect students’ academic experi- 
ence? 

Conclusion 

34) What are your biggest challenges as the High School Program Director? 

35) What have been your biggest successes to date as the High School Program Director? 

36) Is there anything you’d like to share about your role or SEED that we have not dis- 
cussed so far? 
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SEED Evaluation 

Interview Protocol for SEED DC Director of College Counseling 


Introduction 

• Thank respondent for her time. 

• Interviewer(s) introduces self and study. 

• Describe purpose of interview. (Mention follow-up to November 2012 site vis- 
it/interview.) 

• Confirm time allotted for interview. 

• Ask for permission to record the interview. 

Roles and duties 

1) We want to make sure we understand all of your roles and responsibilities at the school. 

a. Please describe your roles and responsibilities for us. 

2) In terms of reporting structures: 

a. Who is your immediate supervisor? 

b. Which staff members report to you? 

c. How large is the College Counseling department? 

3) We’ve heard a little bit about the role of the Programmatic Leadership Team (PLT), but 
we’d like to hear about it from your perspective. 

a. How do you use the PTL to support your work? 

b. How does the PLT support your work? 

SEED’S postsecondary goals 

4) Does SEED have a specific philosophy about postsecondary readiness, access and suc- 
cess? 

a. Please cite. 

5) What are the school’s postsecondary goals for students? 

a. Have the goals changed over time? 
i. If yes, why and how? 

6) Is there an expectation that all students will attend college? 

a. Why or why not? 

7) How many students take the PSAT and/or the SAT/ACT in preparation for applying to 
college? 
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8) When does the college counseling discussion begin? 

a. What types of college advising activities target: 

i. Middle school students 

1 . Specify by grade 

ii. High school students 

1. Specify by grade 

9) Do teaching staff allow the counseling staff to meet with students in their classes to dis- 
cuss college preparation activities? 

a. If yes: 

i. How often do you meet with students in classroom settings? 

ii. Which teachers (subject areas) have been most receptive to allowing 
you to meet with students? 

iii. What types of activities do you lead in the classroom setting? 

1 0) Are Student Life staff expected or instructed to talk about college and college-going in 
a certain way? 

a. How does Student Life programming support SEED’S college-going goals? 

i. At the middle school level 

ii. At the high school level 

College match approach 

11) Please tell us about SEED’S philosophy and approach to college match. 

a. Why focus on match? 

b. Did specific data drive this decision? 

12) How does the match focus influence the colleges that students apply to? 

a. Please describe the development of the tiered list of match colleges approved 
by SEED. 

i. What criteria are used to develop the list? 

ii. How do you work with the CTS team to develop the list? 

iii. Does the tiered list overlap with the Barron’s selectivity categories? 

1. If yes, how? 

13) How has school staff reacted to the focus on match? 

a. Has there been any opposition? 

b. If yes, what does this look like? 

14) Has the focus on match changed counselors’ advising practice? 

a. How? 
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15) Describe how college counseling staff uses the tiered list to engage students in the col- 
lege search process. 

16) When discussing the concept of match with students. . . 

a. Are there specific issues that you raise with students to help them think about 
finding a college that is a good match for them? 

b. Are there specific issues that students identify as limiting their college options? 

1 7) Are there consequences associated with a student choosing to enroll in a college that is 
not on the tiered match list? 

a. How have students and families responded to these consequences? 

18) When does the match-specific discussion begin with students? 

a. In middle school 

b. High school 

c. During the application stage 

The search and application process 

1 9) Describe how you work with upperclassmen. 

a. Juniors 

b. Seniors 

20) How would you describe seniors’ understanding of the college application process? 

a. Probes: 

i. Typically, what do they know? 

ii. Typically, what don’t they know? 

21) What types of activities do students request help with the most ? 

a. Probes: 

i. Going on college tours 

ii. Identifying match/best fit colleges 

iii. Discussing a liberal arts education 

iv. Essay writing 

v. Assembling and posting college applications 

vi. Identifying scholarships 

vii. FAFSA assistance 

viii. Decision-making about college offers 

ix. Other 
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22) What types of activities do students request help with the least ? 

a. Probes: 

i. Going on college tours 

ii. Identifying match/best fit colleges 

iii. Discussing a liberal arts education 

iv. Essay writing 

v. Assembling and posting college applications 

vi. Identifying scholarships 

vii. FAFSA assistance 

viii. Decision-making about college offers 

ix. Other 

23) What selectivity tiers of colleges are SEED students applying to and enrolling in? 

a. Probe: 

i. Example: Using Barron’s selectivity tiers (most competitive, highly 
competitive, etc.) 

24) Typically, what factors have the most influence on where students choose to apply to 
college? 

a. Probes: 

i. Financial aid package/scholarship support 

ii. Academic reputation 

iii. Availability of chosen major 

iv. Closeness to home or distance from home 

v. Family reasons/obligations (specify) 

vi. Other 

College acceptances and decision-making 

25) Do students experience any pressure to accept offers of admission early (before the na- 
tional May 1 st deadline)? 

a. What advice do you give them for how to address this? 

b. Do you contact any colleges/fmancial aid offices directly, on behalf of students 
on your match list? 

26) What factors most influence match students’ decisions about which college to attend ? 

a. Financial aid package 

b. Scholarship support 

c. Academic reputation 

d. Availability of chosen major 

e. Closeness to home or distance from home 

f. Family reasons (specify) 

g. Other 
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Parent outreach 

27) Let’s discuss your work with parents. 

a. In your opinion, what makes SEED attractive to parents? 

b. What does parent outreach from your office look like? 

c. What is the nature of your conversations with parents? 

i. What are they most interested in knowing? 

ii. Are there common questions that parents ask? 

1. Specify. 

d. Do you work with parents on completing the FAFSA? 

i. What are their concerns about completing the FAFSA and financial aid 
in general? 

e. Do you work with any parents to help them evaluate their child’s offers of ad- 
mission? 

i. What main things do you discuss with them? 

f. What’s been your biggest challenge working with parents? 

i. Have you figured out how to overcome it? How? 

College data 

28) Fet’s discuss your work with the College Transition and Support team at the Founda- 
tion. 

a. How would you describe the difference in the goals and the focus of the Col- 
lege Counseling Department and the CTS team? 

i. Is the difference clear? 

b. Do you meet regularly with the CTS staff? 

i. If yes, how often? 

c. Are there activities that allow you to work closely together? 

d. When you meet, what types of topics do you discuss? 

29) Fet’s discuss the college application, enrollment and completion data that SEED col- 
lects. 

a. What systems are in place at the school building level to track college applica- 
tion, enrollment and completion data? 

i. Describe in detail. 

b. What systems are in place at the Foundation level to track college application, 
enrollment and completion data? 

c. How do these data drive changes in the school’s college advising practices? 

Conclusion 

30) What specific supports are in place for students who do not plan to attend college? 
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3 1 ) In general, do you feel that SEED is successful in its college -preparatory mission? 

a. Are there areas in which you’d like the school to improve? 

32) What are your biggest challenges as Director of College Counseling? 

a. What are the key challenges you expect the department to face in the future? 

33) What have been your biggest successes/accomplishments to date as Director of College 
Counseling? 

34) Is there anything we haven’t talked about that would be important for us to know about 
the College Counseling department and its progress to date? 
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SEED Evaluation 

Student Focus Group Protocol: High School 


Introduction (read official introduction and assent language) 

Application to SEED 

1 ) Do you remember applying to SEED? 

a. If so, was it your decision to apply to SEED or was it your parents’ or guardi- 
ans’ decision? 

i. How did you or your family hear about the school? 

b. Do you remember why you applied to SEED? 

i. If so, why? 

ii. Were there other schools that you also applied to? 

c. Did you want to come to the SEED school when you applied? 

d. Why did you enroll? 

Academics 

2) What do you think of the classes at SEED? 

a. Are they hard, in general? 

b. Do teachers give you a lot of homework? 

c. Do you like being in classes that are all girls or all boys? 

i. Why or why not? 

d. Are any of you taking AP classes? 

i. Which classes are you taking? 

ii. How/why did you decide to take these classes? 

iii. Did you have to apply? 

iv. Do you plan to take the AP test(s) next year? 

3) When do you usually do your homework? 

a. Do you feel like you have enough time to get all of your work done? 

b. Are there adults that help you with your homework? 

c. Do you work with your classmates on your homework? 

4) Do any of you participate in tutoring? 

a. How did you find out about tutoring? 

b. What kinds of things do you do with your tutor? 

c. Do you find tutoring helpful? 

5) What do you think of the teachers at SEED, in general? 

a. Do they explain things clearly? 
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b. Do you feel like you can ask them for help if you have a problem in school? 

c. Can you go to them if you are having any personal problems or challenges? 

d. Do you meet with them outside of class time? 

e. Do they talk to you about your future goals? 

College 

6) Do you plan to go to college after high school? 

a. How did you come to this decision? 

7) How is SEED preparing you for college? Probe: 

a. Academically? 

b. Socially? 

8) Are any of you applying now? 

a. If not, 

i. How do you think SEED will help you during this process? 

ii. Has anyone started to talk to you about what this process involves? 

iii. Do you ever go to the College Cafe? Probe: 

1 . Who do you work with when you go there? 

2. What types of things do you do there? 

b. If so, probe: 

i. What schools are you applying to? Probe: 

1 . How did you decide this? 

2. Have any SEED staff been involved in helping you make these 
decisions? 

a. If so, which ones? 

ii. What other adults are working with you? 

1 . What are they helping you with? 

2. When do they help you? 

iii. Do you ever go to the College Cafe? Probe: 

1 . Who do you work with when you go there? 

2. What types of things do you do there? 

iv. When do you work on your applications? 

v. Are there other events or workshops that SEED puts on for you or your 
parents? 

9) Did any of you participate in Junior Seminar last year? 

a. If so, probe: 

i. What types of things did you cover during this time? 

ii. Did you find it helpful? 
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10) Are any of you participating in Senior Seminar right now? 

a. If so, probe: 

i. What types of things did are you covering? 

ii. Are you finding it helpful? 

11) How many of you have taken. . . 

a. The PSAT? 

i. How did SEED prepare you for this? 

ii. Were you required to take it? 

b. The SAT? 

i. How did SEED prepare you for this? 

ii. Were you required to take it? 

c. The ACT? 

i. How did SEED prepare you for this? 

ii. Were you required to take it? 

12) Do you feel like you will be ready for college when you graduate? 

a. If so, probe: 

i. What were the most helpful things that SEED did to prepare you? 

b. If not, probe: 

i. Why do you feel unprepared? 

ii. How could SEED have done a better job preparing you? 

13) Do you expect to stay in touch with SEED while you are in college? 

a. Are you aware of the type of supports SEED offers its graduates? 

Boarding experience 

14) How you do you like living in the dorms? Probe: 

a. What are the best parts of going to a boarding school? 

b. What are the biggest challenges? 

15) What is your relationship with your RA like? probe: 

a. What kinds of things do you normally do with your RA? 

b. Do they help you with your homework? 

c. Do they teach you other things? 

d. Do you feel like you can ask her/him for help if you have a problem in school? 

e. Can you go to her/him if you are having any personal problems or challenges? 

f. Do they talk to you about your future goals? 
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Student Life 

16) How do you spend your time after school? Probe: 

a. Do you study? 

b. Do you participate in any Sparks? 

c. Do you participate in any sports teams? 

d. Do you participate in HALLS lessons? 

i. How often do these happen? 

ii. What things are usually covered during these lessons? 

e. Do you ever participate in community service projects? 

f. Do you belong to any other types of clubs or organizations? 

g. What else do you do after school? 

17) What is your relationship with the Life Skills Counselors (LSCs) like? Probe: 

a. What kinds of things do you normally do with LSCs? 

b. Do they help you with your homework? 

c. Do they teach you other things? 

d. Do you feel like you can ask them for help if you have a problem in school? 

e. Can you go to them if you are having any personal problems or challenges? 

f. Do they talk to you about your future goals? 

18) Do you feel like SEED is teaching you useful life and social skills? 

a. If so, probe: 

i. What skills do they focus on? 

ii. When do you learn these skills? 

iii. How do you think these skills will help you in your future? 

b. If not, probe: 

i. Do you think they are trying to do this and are just not doing a good 
job? 

ii. If so, how could they do a better job? 

External opportunities 

1 9) Have you worked with the External Opportunities Office? 

a. If so, how have you worked with them? 

b. Have they helped you find any summer activities or internships? 

i. If so, probe: 

1 . What have you done? 

2. How did you choose this opportunity? 

3. How was SEED involved in preparing you for this opportunity? 
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20) Have you gone on any field trips while you have been at SEED? Probe: 

a. To colleges? 

b. Where else? 

Weekend transition 

21) What is it like to go home on the weekends? Probe: 

a. Do you feel like it is difficult transition? 

i. Why or why not? 

22) What do you do on the weekends? 

23) What is it like coming back to SEED on Sundays? 

Friends 

24) Who are your closest friends? Probe: 

a. Do they also go to SEED? 

i. If not, probe: 

1 . What is it like seeing your friends on the weekends? 

2. Do you talk about school with them? 

a. If so, do you thi nk you’re having a similar experience 
to them? 

i. Why or why not? 

ii. If yes probe: 

1 . Do they live near you at home? 

25) Do your closest friends want to go to college? 

Other 

26) Besides teachers, RAs, and LSCs, are you close with any other adults at SEED? 

a. If so, probe: 

i. Who are you close to? 

ii. What kinds of things do you do with this person/these people? 

iii. Do you feel like you can ask them for help if you have a problem in 
school? 

iv. Can you go to them if you are having any personal problems or chal- 
lenges? 

v. Do they talk to you about your future goals? 

b. If not, probe: 

i. Why not? 
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27) Do you have a lot of free time at SEED? 

a. If so, probe: 

i. When is this free time? 

ii. What do you do during free time? 

iii. Do you wish you had less free time? 

b. If not, probe: 

i. Do you feel like you ever have time to yourself? 

ii. Would you want more free time? 

iii. Do you ever feel too busy? 

c. How do you know what activities are when? 

d. Do you think SEED is doing a good job of teaching you how to manage your 
time? 

28) How is high school different than ninth grade? Probe: 

a. Academically? 

b. Socially? 


Closing 

29) What are your top three favorite things about SEED? 

30) What are your three least favorite things about SEED? 

31) How do you think SEED could do a better job 

a. Helping you learn? 

b. Providing you with activities after school? 

c. Making the dorms a good place to live? 

32) Is there anything else that we haven’t asked you about that we should know? 
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Appendix C 

The SEED DC Lottery Process 




As is common among charter schools nationwide, SEED DC conducts a lottery each spring to 
determine which new students are offered the opportunity to enroll in the school the following 
fall. Entry is typically limited to one grade level, after which students already enrolled at SEED 
can freely progress into subsequent grades without having to “win” a seat. SEED’S grade of en- 
try changed during the study period. Students who applied to start at SEED in the 2007-2008 
and 2008-2009 school years applied as sixth-graders, vying for seats in the seventh-grade class. 
SEED then decided to move toward a sixth-grade entry point in response to a growing trend 
among elementary schools of graduating students after fifth grade instead of sixth. To allow for 
a transitional period, students could apply for seats in both the sixth- and seventh-grade classes 
in the 2009-2010 and 2010-201 1 school years. The 201 1-2012 school year marked the first year 
in which students could begin at SEED only as sixth-graders. 

Having one point of entry means that open seats, even if they occur in the higher 
grades, are almost always filled by the incoming middle school applicants. For example, if an 
eleventh-grader left SEED during the 2014-2015 school year, the student’s “seat” would be 
filled by an incoming sixth-grader participating in the lottery for the 2015-2016 school year. 1 

Any District of Columbia resident who was bom on or after February 1,12 years before 
the lottery, is eligible to attend SEED. Families need only complete an application to be includ- 
ed in the lottery. In the past, families were required to attend the lottery in person, but in 2008 
SEED made their presence optional. 

SEED uses a bingo ball and cage system to select students. Students are assigned a ran- 
dom number that corresponds to one of the bingo balls, and the director of enrollment reads the 
numbers as they drop from the cage. Separate lotteries are held for boys and girls, and the num- 
ber of seats being filled varies from year to year based on SEED’S enrollment needs and pre- 
dicted attrition. Once available seats are filled, the order in which students’ numbers are chosen 
detennines their places on the waiting list. Students who apply to SEED after the deadline are 
placed at the end of the waiting list in the order in which their applications are received. 

Students with siblings already enrolled in SEED receive preference, and a student with 
a sibling participating in the lottery in the same year is admitted if the sibling is offered a place. 
In the lotteries for the 2006-2007 through 2008-2009 school years, applicants with siblings al- 
ready enrolled at SEED participated in the lottery but were later (nonrandomly) pushed to the 
top of the list. In the 2009-2010 lottery, students with SEED siblings no longer participated in 
the lottery; they were instead offered the opportunity to attend SEED. 


'in some infrequent cases, when the entering grade, currently sixth grade, is at capacity in terms of staff, 
materials, class size, and so on, SEED enrolls students in later grades. If this occurs, students are still selected 
randomly based on the lottery in which they participated. 
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Students selected in the lottery must accept or reject the offer by a specified deadline. 
SEED follows up with families who do not respond by the deadline, giving them two business 
days to respond to the reminder. After this follow-up, the director of enrollment management 
fills open seats by moving down the waiting list. Families are given the same two-business-day 
period to respond to the offer before SEED moves on to the next student. The director of en- 
rollment will make very infrequent exceptions if the family has a compelling reason for failing 
to respond to the offer, for example, if the responding parent was incarcerated. SEED DC con- 
tinues to fill seats until early October. 

All students who are selected in the lottery and are offered a seat at SEED on lottery 
day are invited to attend a weeklong new-student orientation during the summer preceding their 
enrollment at SEED. A select number of students from the waiting list are also invited in antici- 
pation of a certain number of students changing their mind over the summer. Attendance at the 
orientation is not required but is strongly encouraged. 

Figure C. 1 shows how the study sample progressed through this lottery and enrollment 
process, beginning with 919 total applicants and 499 students being offered the opportunity to 
enroll at SEED. 
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SEED Evaluation Final Report 

Figure C.l 


Flow of Study Sample Through SEED Application Process 



SOURCES: MDRC’s calculations using SEED DC lottery data and Office of the State Superintendent of Education 
(OSSE) files on student enrollment and graduation. 

NOTE: Tn the 2006-2007 through 2008-2009 lotteries, 39 siblings of students already enrolled in SEED were 
technically included in the lottery and then removed for automatic enrollment. These individuals are not counted as 
lottery participants; they are counted among the applicants automatically given the opportunity to enroll. Since their 
removal was based on a baseline characteristic, it does not affect the randomness of assignment of the remaining 
participants; that is, their removal does not disturb the internal validity of the remaining lottery sample. 
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Appendix D 

The Analytic Approach 




Impact Design 

A student-level randomized experiment was used to evaluate SEED’S impact on the achievement 
and engagement outcomes of students offered the program compared with how they would have 
perfonned otherwise. During the study period, a greater number of rising sixth-grade students 
applied to SEED than could be served, so lotteries were conducted each year to determine who 
would be offered an opportunity to enroll. 1 A subset of the applicants were randomly offered the 
opportunity to fill SEED’S approximately 60 school slots, constituting the SEED group. The re- 
mainder of the applicants were not offered admission to SEED and therefore constitute the con- 
trol or “business as usual” condition — what would occur in the absence of SEED. 2 As described 
in Appendix C, students who were siblings of current SEED students did not participate in the 
lottery; they were automatically admitted to SEED and are not part of the study sample. Since 
random assigmnent creates comparable groups of students when samples are large, 3 any differ- 
ence in post-random assigmnent outcomes between those who were offered the opportunity to 
enroll in SEED and those who were not represents the impact of SEED. 


Statistical Analysis of Impacts 

SEED’S impact was estimated by comparing the outcomes of students assigned to SEED and 
those of the non-SEED control group. The following statistical model was used: 

J s 

Y i - ^ a j ' hj + fiTi + ^ Ys ' %si + £ i 
j = i ;'= i 


where: 

Yj = outcome for student i 

T, = indicator of treatment group membership, equal to 1 if student i was randomly assigned to 
SEED participation and 0 otherwise 


To achieve equal numbers of males and females in their entering seventh-grade class, SEED DC con- 
ducts one lottery for boys and one lottery for girls. 

2 Some of the students who are not admitted to SEED initially are placed on a waiting list. When students 
who win the lottery choose not to enroll in the school, students may be admitted to SEED from the waiting list. 
MDRC researchers reviewed SEED waiting lists and confirmed that during the study period, students were 
admitted to SEED from the waiting list in the random order determined on lottery day. Accordingly, any stu- 
dent from the waiting list who was offered the opportunity to enroll in SEED before the fall of the following 
school year is considered a SEED group member. (This approach is similar to the one taken by Tuttle and col- 
leagues in their 2013 report on KIPP charter schools.) 

3 As shown in Table 3.1, students in the SEED group are more likely not to qualify for free or reduced- 
price lunch and have slightly higher test scores. In accordance with What Works Clearinghouse recommenda- 
tions, these characteristics are included as co variates in the impact model. 


99 



Iy = a set of j random assignment block indicators, equal to 1 if student i is in random assign- 
ment block j and 0 otherwise. A student’s block is defined based on her or his cohort (year of 
random assignment) and gender at the point of random assignment. These blocks were included 
in the model to capture a central feature of the research design (lotteries by school and gender 
within year). 

X sl = a set of s pre-random assignment characteristics for student i that are associated with stu- 
dent outcomes. These characteristics include standardized test scores from students’ baseline 
year and indicators of family income. 

8,- = a student-level random error term. 

In this model, [1 represents the estimated impact of SEED on the outcome of interest 
(T/). Because the statistical analysis treats random assignment blocks as fixed effects, and [1 is 
estimated as a fixed coefficient rather than a random one, [1 should be interpreted as the estimat- 
ed impact of SEED for the average student in the set of schools where the study will take place. 
This approach is taken because the study sites are not a random sample of a larger population of 
schools or districts. Consequently, the evaluation results will not be generalizable to other 
schools or students. 

(3 is also an “intent-to-treat” estimate of the SEED school program impact. Some stu- 
dents assigned to SEED will not enroll. Thus, the main evaluation findings represent the esti- 
mated impact of offering students the opportunity to enroll in SEED (“intent to treat”), rather 
than the impact of SEED on students who actually attended (“treatment on the treated”). It was 
expected that SEED participation among the program group would initially be high but would 
decrease over time. Thus, in a quasi-experimental analysis, an instrumental variables approach 
was used to estimate the effects of actually enrolling in SEED DC (see Appendix E). 4 

Sample Retention and Missing Data 

Much of the impact study relies on existing administrative records from SEED, such as 
the lottery application and assignment offering, and from the Office of the State Superintendent 
of Education (OSSE), such as state standardized test scores and graduation records. After a 
small decrease (8 percent) in the sample based on the researchers’ inability to locate all lottery 
applicants in OSSE student records, there was a yearly attrition rate between 5 percent and 10 
percent. Despite the yearly attrition, the characteristics of the sample and its internal validity 
appear to have been maintained. For example, Table D.l presents the baseline characteristics of 
the first two cohorts, and Table D.2 shows the same characteristics for the sample members 
from these cohorts for whom graduation data were available six years later. 


4 See Schochet and Chiang (2009). 
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SEED Evaluation Final Report 
Table D.l 

Baseline Characteristics of SEED Lottery Participants, Cohorts 1 and 2 


Characteristic 

SEED 

Group 

Non-SEED Estimated 

Group Difference 

P-Value for 

Estimated 

Difference 

Race (%) 





Black 

100.0 

100.0 

0.0 

1.000 

Female (%) 

43.8 

43.8 

0.0 

1.000 

Economic indicator (%) 





Eligible for free/reduced-price lunch 

NA 

NA 

NA 

— 

Direct Certification status' 1 

NA 

NA 

NA 

— 

Special education status (%) 

12.0 

21.1 

-9.1 

0.120 

Standardized math score 

11.53 

-5.52 

17.05 

0.202 

At or above proficiency level in math (%) 

NA 

NA 

NA 

— 

Standardized reading score 

9.87 

-9.80 

19.67 

0.176 

At or above proficiency level in reading (%) 

NA 

NA 

NA 

— 

Sample size 

121 

93 




SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) 
files on student enrollment and state test scores from the 2006-2007 and 2007-2008 school years. 

NOTES: NA = not available. 

Cohorts 1 and 2 consist of 214 students in the study who were sixth-graders in the springs of 
2007 and 2008, respectively. 

Values for SEED group members and non-SEED group members are the simple means or 
percentages. Values for the difference between SEED group members and non-SEED group 
members are obtained from a regression of a given baseline characteristic on a series of indicator 
variables that identify each lottery, a covariate indicating the probability that a student would win 
a spot on lottery day, and an indicator variable that equals 1 for lottery winners and 0 for lottery 
losers. The coefficient on the latter indicator variable equals the difference in the baseline 
characteristic between SEED group members and non-SEED group members. The value for non- 
SEED group members equals the corresponding value for SEED group members minus the 
estimated difference between SEED group members and non-SEED group members. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a Direct Certification is automatic qualification for free lunch because the individual is in foster 
care, is homeless, or is living in a household receiving benefits from the Supplemental Nutrition 
Assistance Program (SNAP), the Food Distribution Program on Indian Reservations (FDPIR), or 
Temporary Assistance for Needy Families (TANF). 
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SEED Evaluation Final Report 
Table D.2 


Baseline Characteristics of SEED Lottery Participants in 
Cohorts 1 and 2 for Whom Graduation Data Are Available 


Characteristic 

SEED 

Group 

Non-SEED 

Group 

Estimated 

Difference 

P- Value for 

Estimated 

Difference 

Race (%) 





Black 

100.0 

100.0 

0.0 

1.000 

Female (%) 

46.7 

46.7 

0.0 

1.000 

Economic indicator (%) 





Eligible for free/reduced-price lunch 

NA 

NA 

NA 

— 

Direct Certification status 3 

NA 

NA 

NA 

— 

Special education status (%) 

10.8 

13.6 

-2.8 

0.662 

Standardized math score 

15.98 

3.07 

12.90 

0.545 

At or above proficiency level in math (%) 

NA 

NA 

NA 

— 

Standardized reading score 

12.37 

-13.10 

25.47 

0.235 

At or above proficiency level in reading (%) 

NA 

NA 

NA 

— 

Sample size 

75 

58 




SOURCES: MDRC’s calculations use SEED DC school application data for students in 2006-2007 
through 2011-2012 and Office of the State Superintendent ofEducaton (OSSE) files on student 
enrollment and state test scores from the 2006-2007 to 2013-2014 school years. 

NOTES: NA = not available. 

Cohorts 1 and 2 consist of 133 students in the study who were sixth-graders in the springs of 2007 
and 2008, respectively. 

Values for SEED group members and non-SEED group members are the simple means or 
percentages. Values for the difference between SEED group members and non-SEED group members 
are obtained from a regression of a given baseline characteristic on a series of indicator variables that 
identify each lottery, a covariate indicating the probability that a student would win a spot on lottery 
day, and an indicator variable that equals 1 for lottery winners and 0 for lottery losers. The coefficient 
on the latter indicator variable equals the difference in the baseline characteristic between SEED 
group members and non-SEED group members. The value for non-SEED group members equals the 
corresponding value for SEED group members minus the estimated difference between SEED group 
members and non-SEED group members. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a Direct Certification is automatic qualification for free lunch because the individual is in foster 
care, is homeless, or is living in a household receiving benefits from the Supplemental Nutrition 
Assistance Program (SNAP), the Food Distribution Program on Indian Reservations (FDPIR), or 
Temporary Assistance for Needy Families (TANF). 
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Appendix E 

Estimated Effects of Enrolling in SEED: 
A “Treatment-on-the-Treated” Analysis 




As described in Appendix D, the primary research design takes advantage of SEED lotteries to 
identify a set of students who randomly “won” the opportunity to attend SEED and a set of stu- 
dents who randomly “lost” and did not receive that opportunity. Because of this random as- 
signment, any difference in students’ future outcomes can be attributed to the opportunity to 
attend SEED, otherwise known as the “intent-to-treat” effect of SEED. But receiving an offer to 
enroll in SEED is not the same as actually enrolling; some lottery winners decide to attend a 
different school. As shown in Table E.l, roughly three-quarters of SEED lottery winners initial- 
ly enroll in SEED, and the proportion of this group who remain enrolled in SEED decreases 
each year. 

Because those who have the opportunity to attend SEED but choose not to are self- 
selected, rather than randomly selected, simply analyzing the outcomes among those who attend 
SEED and those who do not is subject to bias. Therefore, to estimate the effect of actually enrol- 
ling in SEED, this study employs what is known as a “treatment on the treated” approach using 
a standard application of instrumental variables (IV) analysis. 1 In any IV analysis, the key as- 
sumption involves the selection of an instrumental variable (for example, winning or losing the 
lottery) that is related to the outcome only through its relationship to the treatment variable (for 
example, enrolling in SEED). In this analysis the instruments include interactions between each 
lottery fixed effect (gender and year of random assignment) and the treatment dummy (winning 
or losing) for that lottery; these instruments are used to predict actual SEED enrollment. Be- 
cause enrollment is a function of winning a lottery, which is a random process, the instrument 
fulfills the key assumption and is valid, and therefore allows the estimation for the full sample 
of the average treatment effect of actually enrolling in the SEED program. These effects of en- 
rolling in SEED are roughly 20 percent larger than the intent-to-treat effects and are presented 
in Table E.2. 


'Gennetian, Morris, Bos, and Bloom (2005). The research team chose not to employ analytic models that 
have been employed in the past to estimate the yearly effects of receiving varying amounts of exposure to 
SEED (one year, two years, etc.). To be accurate these models require that researchers make the strong as- 
sumption of a linear relationship between years of exposure and impacts. Given that we are following students 
through middle and high school, a time of great, dynamic change in students, the assumption of linearity is not 
realistic. 
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Table E.l 


Program Group Enrollment in SEED by Cohort and Year 



Percentage of SEED Group Enrolled in SEED 


Year 1 

Year 2 

Year 3 

Year 4 

Year 5 

Year 6 

Cohort 1 

73.9 

69.4 

NA 

47.5 

NA 

44.0 

Cohort 2 

92.1 

88.2 

NA 

75.9 

NA 

62.5 

Cohort 3 

68.4 

72.2 

NA 

75.0 

NA 

NA 

Cohort 4 

62.9 

56.3 

NA 

27.6 

NA 

NA 

Cohort 5 

83.8 

80.6 

77.4 

NA 

45.5 

NA 

Cohort 6 

75.0 

60.0 

42.3 

NA 

NA 

NA 

Cohort 7 

80.9 

66.2 

66.7 

NA 

NA 

NA 

Cohort 8 

42.7 

40.0 

NA 

NA 

NA 

NA 


SOURCE: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) 
enrollment data, state test score data and state high school graduation data. 

NOTE: NA = not available. 

This table presents the proportion of students with OSSE follow-up data enrolled in SEED 
over time. 
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Table E.2 

Estimated Effects of Enrolling in SEED, 
Cohorts 1 and 2 


Characteristic 

Target 

SEED 

Enrollees 

Non-SEED 

Group 

Counterparts 

Estimated 

Difference 

P- Value for 

Estimated 

Difference 

First year 

Standardized math score 

0.26 

-0.10 

0.36 * 

0.060 

Standardized reading score 

0.03 

-0.03 

0.06 

0.751 

Second year 

Standardized math score 

0.32 

-0.21 

0.53 *** 

0.006 

Standardized reading score 

0.07 

-0.02 

0.09 

0.640 

Fourth year 

Standardized math score 

0.51 

0.26 

0.26 

0.877 

Standardized reading score 

0.24 

0.32 

-0.08 

0.711 

Four-year high school graduation (%) 

69.3 

74.1 

-4.8 

0.592 

Sample size 

121 

93 




SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) state 
test scores from the 2007-2008 to 2011-2012 school years. 


NOTES: Cohorts 1 and 2 consist of 214 students in the study who were sixth-graders in the springs of 
2007 and 2008, respectively. 

This table presents estimated effects for students who have follow-up data. Appendix E describes how 
values in the columns labeled "Target SEED Enrollees" and "Estimated Difference" are estimated. Each 
value for non-SEED group counterparts equals the corresponding value for target SEED enrollees minus 
the estimated difference. 

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are indicated 
as: *** = 1 percent; ** = 5 percent; * = 10 percent. 


107 








Appendix F 

The SEED Student Survey 




Design and Fielding 

In addition to assessing the impact of SEED DC on students’ academic outcomes, like District 
of Columbia Comprehensive Assessment System (DC CAS) test scores and high school grad- 
uation, the SEED evaluation aimed to understand its effects on a variety of nonacademic out- 
comes, such as metacognitive skills (self-control and “grit”), attitudes, health, and both positive 
and negative youth behaviors. SEED DC’s boarding school model positions it well to affect 
such outcomes not only because the school controls students’ environments for longer periods 
of time than traditional schools, but also because its Student Life programming includes explicit 
life skills instruction and character education. 

To capture SEED DC’s impact on these measures, the research team fielded a survey to 
both program group and non-program group students in cohorts 1 and 2. (The full survey ap- 
pears at the end of this appendix.) These cohorts were selected because, if students in these co- 
horts progressed one grade level per school year, they would have been in their first post-high 
school year and seniors in high school, respectively, at the time of fielding. These older grades 
provided the best opportunity to understand students’ attitudes about attending college as well 
as the longer-term impact of SEED on both positive and negative youth behaviors. 

Because the survey was fielded to both program group students and non-program group 
students, it also served as an important tool in understanding the differences in experiences be- 
tween both research groups. Students were asked questions about their school climate, home 
environment during the week, courses and activities at school, and access to college preparatory 
activities. 

The survey was created using a variety of validated external scales to understand atti- 
tudes, behaviors, and opinions. Table F. 1 provides the Cronbach’s alpha values for these scales 
(indicators of their reliability). Original questions and scales were also included to understand 
more about the courses, activities, and college preparation students experienced. More detailed 
infomiation on the scales used and related outcome measures is provided below. 

The research team partnered with Decision Information Resources to field the survey. 
Students had the opportunity to complete it by phone or on the Internet and were compensated 
for their time with a small monetary reward. The full student survey in Computer-Assisted Tel- 
ephone Interview (CATI) questionnaire format is available upon request. 
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Table F.l 


Cronbach's Alpha Values for Survey Constructs 


Scale 

Cronbach's Alpha 

Study skills 

0.76 

Organizational skills 

0.78 

Self-control 

0.78 

Grit 

0.68 

Recent frequency of risky behavior 

0.82 

Overall engagement in risky behavior 

0.79 

School comfort 

0.79 

Classroom order 

0.81 

Peers' academic motivation 

0.86 

Caring adult at school 

0.90 

School-wide future orientation 

0.90 

Access to college preparatory activities 

0.94 

Order of home during the week 

0.78 

Caregiver support during the week 

0.87 


SOURCE: MDRC calculations based on survey data. 


Detailed Information on Scales Used 

Access to College Preparatory Activities Scale 

The access to college preparatory activities scale consists of six survey questions from 
the University of Chicago Consortium on School Research (UChicago Consortium) 2012 My 
Voice, My School Survey. 1 The questions ask respondents to indicate how often they discussed 


University of Chicago Consortium on School Research (2012). 
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various topics related to preparation for college, including different admissions requirements 
among four-year colleges, how to decide which college to attend, the likelihood of being ac- 
cepted at different types of colleges, what ACT/SAT scores are needed to get into certain col- 
leges, opportunities to attend out-of-state colleges, and how to pay for college. Responses were 
given on an ordinal scale (“did not discuss,” “discussed briefly,” and “discussed in depth”). Re- 
sponses of “did not discuss” were coded as 1, “discussed briefly” as 2, and “discussed in depth” 
as 3. Individuals’ responses to the each of the six questions were averaged to create a construct 
outcome measure for each individual. Higher scores indicate more access to college preparatory 
activities. 

School Comfort Scale 

The school comfort scale is made up of five survey questions that ask respondents to 
indicate how much they agree with a series of statements about the 2012-2013 school year or 
the last year they were in school. 2 Statements concern feeling safe at school, enjoying going to 
school, switching schools if the respondent had had the ability to do so, feeling accepted at 
school, and fitting in at school. Responses were given on an ordinal scale (“strongly disagree,” 
“disagree,” “agree,” and “strongly agree”). Responses of “strongly disagree” were coded as 1, 
“disagree” as 2, “agree” as 3, and “strongly agree” as 4. Individuals’ responses to each of the 
five items were averaged to create a construct outcome measure. Since this scale uses reverse 
wording to phrase statements in both positive and negative voices, several scores were reversed 
so that higher scores indicate higher levels of comfort. 

Classroom Order Scale 

The classroom order scale comprises three survey questions from the UChicago Con- 
sortium 2012 My Voice, My School Survey. 2 The questions ask respondents to indicate how 
much they agree with a series of statements about the 2012-2013 school year or the last year 
they were in school. Statements include being distracted by students acting out, classes being 
out of control, and having classmates who did not behave the way teachers wanted them to. Re- 
sponses were given on an ordinal scale (“strongly disagree,” “disagree,” “agree,” and “strongly 
agree”). Responses of “strongly disagree” were coded as 1, “disagree” as 2, “agree” as 3, 
“strongly agree” as 4. Individuals’ responses to each of the three items were averaged to create a 
construct outcome measure. Since this scale includes statements in only the negative voice 
(statements about disorderly classrooms), all scores were reversed so that higher scores indicate 
higher levels of classroom order. 


2 These questions are based on questions from the UChicago Consortium 2012 My Voice, My School Sur- 
vey (University of Chicago Consortium on School Research, 2012). 

’University of Chicago Consortium on School Research (2012). 
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Peers’ Academic Motivation Scale 


The peers’ academic motivation scale consists of four survey questions from the UChi- 
cago Consortium 2012 My Voice, My School Survey. 4 The questions ask respondents how 
many of their peers in the 2012-2013 school year or the last year of school they attended felt it 
was important to come to school every day, felt it was important to pay attention in class, felt it 
was important to do homework, and tried hard to get good grades. Responses were given on an 
ordinal scale (“none,” “a few,” “about half,” and “most”). Responses of “none” were coded as 
1, “a few” as 2, “about half’ as 3, and “most” as 4. Individuals’ responses to each of the four 
items were averaged to create a construct outcome measure. Higher scores indicate more aca- 
demic motivation among peers. 

Caring Adult at School Scale 

The caring adult at school scale consists of six survey questions based on the California 
Healthy Kids Survey. 5 The questions ask respondents to indicate how much they agree with a 
series of statements about the school they attended in 2012-2013 or the last year they were in 
school. Statements concern having had teachers or adults at school who really cared about the 
respondent, told the respondent when he or she did a good job, noticed when the respondent 
was not there, always wanted the respondent to do his or her best, listened when the respondent 
had something to say, and believed that the respondent would be a success. Responses were 
given on an ordinal scale (“not at all true,” “a little true,” “pretty much true,” and “very much 
true”). Responses of “not at all true” were coded as 1, “a little true” as 2, “pretty much true” as 
3, and “very much true” as 4. Individuals’ responses to each of the six items were averaged to 
create a construct outcome measure. Higher scores indicate more supportive caregivers. 

School-Wide Future Orientation Scale 

The school-wide future orientation scale is made up of six survey questions from the 
UChicago Consortium 2012 My Voice, My School Survey. 6 The questions ask respondents to 
indicate how much they agree with a series of statements about the 2012-2013 school year or 
the last year they were in school. Statements included whether students had been at a school 
where teachers made sure that all students were planning for a life after graduation, teachers 
worked hard to make sure that all students were learning, high school was seen as preparation 
for the future, all students were encouraged to go to college, teachers paid attention to all stu- 
dents, and teachers worked hard to make sure that all students stayed in school. Responses were 
given on an ordinal scale (“strongly disagree,” “disagree,” “agree,” and “strongly disagree”). 


4 University of Chicago Consortium on School Research (2012). 

California Department of Education (2015). 

‘’University of Chicago Consortium on School Research (2012). 
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Responses of “strongly disagree” were coded as 1, “disagree” as 2, “agree” as 3, “strongly 
agree” as 4. Individuals’ responses to each of the six items were averaged to create a construct 
outcome measure. Higher scores indicate having a school that is more future-oriented. 

Order of Home During the Week Scale 

The order of home during the week scale comprises 1 1 survey questions adapted from 
items in the Confusion, Order, and Hubbub Scale. 7 The questions ask respondents to indicate 
how much they agree with a series of statements about where they live from Sunday to Thurs- 
day night (the time that SEED students live on campus). Statements concern being able to find 
things when you need them, feeling rushed, being able to stay on top of things, running late, 
being able to talk to others without being interrupted, often having a fuss going on, having plans 
that often do not work out, not being able to hear yourself think, getting drawn into other peo- 
ple’s arguments, having a good place to relax, and having a calm atmosphere. Responses were 
given on an ordinal scale (“very much like where you live during the week,” “somewhat like 
where you live during the week,” “a little bit like where you live during the week,” and “not at 
all like where you live during the week”). Responses of “very much like where you live during 
the week” were coded as 1, “somewhat like where you live during the week” as 2, “a little bit 
like where you live during the week” as 3, and “not at all like where you live during the week” 
as 4. Individuals’ responses to each of the 1 1 items were averaged to create a construct outcome 
measure. Since this scale uses reverse wording to phrase statements in both positive and nega- 
tive voices, several scores were reversed so that higher scores indicate having higher levels of 
order. 


Caregiver Support During the Week Scale 

The caregiver support during the week scale consists of six original survey questions. 
The questions ask respondents to indicate how true they perceive a series of statements to be 
about adults with whom they live from Sunday to Thursday night (the time that SEED students 
live on campus). Statements include whether there is an adult who expects the respondent to 
follow rules, who is interested in the respondent’s schoolwork, who believes that the respondent 
will be a success, who talks with the respondent about his or her problems, who always wants 
the respondent to do his or her best, and with whom the respondent likes to do fun things. Re- 
sponses were given on an ordinal scale (“not at all true,” “a little true,” “pretty much true,” and 
“very much true”). Responses of “not at all true” were coded as 1, “a little true” as 2, “pretty 
much true” as 3, and “very much true” as 4. Individuals’ responses to each of the six items were 
averaged to create a construct outcome measure. Higher scores indicate a greater level of sup- 
port from adults during the week. 


7 Matheny, Wachs, Ludwig, and Philips (1995). 
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Study Skills Scale 

The study skills scale consists of four survey questions from the UChicago Consortium 
2012 My Voice, My School Survey. 8 The questions ask respondents to indicate how much they 
agree with a series of statements about always studying for tests, setting aside time to do home- 
work and study, trying to do well on schoolwork even when it is not interesting, and forgoing 
time with friends when they need to study. Responses were given on an ordinal scale (“strongly 
disagree,” “disagree,” “agree,” and “strongly agree”). Responses of “strongly disagree” were 
coded as 1, “disagree” as 2, “agree” as 3, “strongly agree” as 4. Individuals’ responses to each 
of the four items were averaged to create a construct outcome measure. Higher scores indicate 
better study skills. 

Organizational Skills Scale 

The organizational skills scale comprises three survey questions from the UChicago 
Consortium 2012 My Voice, My School Survey. 9 The questions ask respondents to indicate 
how much they agree with a series of statements about keeping track of long-term assignments, 
managing time well enough to get all work done, and keeping their schoolwork and personal 
life organized. Responses were given on an ordinal scale (“strongly disagree,” “disagree,” 
“agree,” and “strongly agree”). Responses of “strongly disagree” were coded as 1, “disagree” as 
2, “agree” as 3, “strongly agree” as 4. Individuals’ responses to each of the three items were 
averaged to create a construct outcome measure. Higher scores indicate better organizational 
skills. 


Recent Frequency of Risky Behavior Scale 

The recent frequency of risky behavior scale is based on six survey questions from the 
Self-Reported Behavior Index. 10 Respondents are asked to indicate the frequency with which 
they have engaged in specific risky behaviors during the last three months. Behaviors include 
getting into a fight at school or in the neighborhood, having a fight or argument with parents, 
taking something that did not belong to the respondent, skipping school without pennission, 
taking something from a store without paying for it, and hitting someone. Responses were given 
on an ordinal scale that was different than the original used in the Self-Reported Behavior Index 
(“you have never done this,” “you have done this, but not in the last 3 months,” “you did this 1- 
2 times in the last 3 months,” “you did this 3-4 times in the last 3 months,” and “you did this 5 
or more times in the last 3 months”). Responses of “you have never done this” were coded as 1, 
“you have done this, but not in the last 3 months” as 2, “you did this 1-2 times in the last 3 


s University of Chicago Consortium on School Research (2012). 
"’University of Chicago Consortium on School Research (2012). 
10 Brown, Clasen, and Eicher (1986). 
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months” as 3, “you did this 3-4 times in the last 3 months” as 4, and “you did this 5 or more 
times in the last 3 months” as 5. Individuals’ responses to each of the six items were averaged to 
create a construct outcome measure. Higher scores indicate more frequent risky behavior in the 
past three months. 

Overall Engagement in Risky Behavior Scale 

The overall engagement in risky behavior scale comprises four original questions about 
individuals’ involvement in specific risky behaviors since the time of random assignment. The 
questions ask whether the individual has ever been expelled or dismissed from school, has ever 
been in trouble with the police, has ever been arrested or had to go to juvenile court, or has had 
unprotected sex. Responses included “yes” and “no,” “yes” being coded as 1 and “no” as 0. In- 
dividuals’ responses to each of the four items were averaged to create a construct outcome 
measure. Higher scores indicate being more likely to have ever engaged in risky behavior since 
random assignment. 

Self-Control Scale 

The self-control scale uses 13 survey questions that constitute the Brief Self-Control 
Scale. 11 Respondents are asked to indicate how true they perceive a series of statements to be. 
The statements cover topics like breaking bad habits, the ability to refuse things that are not 
good, self-discipline, and thinking through alternatives before acting. Responses were given on 
an ordinal scale (“not at all like you,” “not much like you,” “somewhat like you,” “mostly like 
you,” and “very much like you”). Responses of “not at all like you” were coded as 1, “not much 
like you” as 2, “somewhat like you” as 3, “mostly like you” as 4, and “very much like you” as 
5. Individuals’ responses to each of the 13 items were averaged to create a construct outcome 
measure. Since this scale uses reverse wording to phrase statements in both positive and nega- 
tive voices, several scores were reversed so that higher scores indicate having more self-control. 

Grit Scale 

The grit scale uses seven survey questions from the Short Grit Scale. 12 Respondents are 
asked to indicate how true they perceive a series of statements to be. The statements cover top- 
ics such as whether new ideas distract from previous ones and whether the respondent is a hard 
worker, is diligent, and finishes what he or she begins. Responses were given on an ordinal 
scale (“not at all like you,” “not much like you,” “somewhat like you,” “mostly like you,” and 


'Tangney. Baumeister, and Boone (2004). 
12 Duckworth, Peterson, Matthews, and Kelly (2007). 


117 



“very much like you”). Responses of “not at all like you” were coded as 1, “not much like you” 
as 2, “somewhat like you” as 3, “mostly like you” as 4, and “very much like you” as 5. Individ- 
uals’ responses to each of the seven items were averaged to create a construct outcome measure. 
Since this scale uses reverse wording to phrase statements in both positive and negative voices, 
several scores were reversed so that higher scores indicate having higher levels of grit. 
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About MDRC 


MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to leam not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy are- 
as and target populations. Once known primarily for evaluations of state welfare-to-work pro- 
grams, today MDRC is also studying public school refonns, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 



